Palo Alto, CA, Remote
March 22, 2024
Apply NowJob description
Job Description:
- We are seeking a talented and experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in data engineering with a focus on Microsoft Azure services. The candidate should have a minimum of 10+ years of hands-on experience with Azure Data technologies.
- Azure Data Lake Storage Gen2: Understanding of Azure Data Lake Storage Gen2 and its capabilities for storing large volumes of structured, semi-structured, and unstructured data.
- Azure Data Factory: Proficiency in designing, implementing, and orchestrating data integration workflows using Azure Data Factory. Knowledge of activities, data flows, and integration run times.
- Azure Databricks: Experience in leveraging Azure Data-bricks for big data processing, data engineering, and machine learning tasks. Knowledge of Spark-based data processing and optimization techniques.
- Azure Synapse Analytics (formerly SQL Data Warehouse): Familiarity with Azure Synapse Analytics for data warehousing and big data analytics. Ability to design and implement scalable data warehouse solutions.
- Azure SQL Database: Proficiency in working with Azure SQL Database for relational data storage and management. Knowledge of performance optimization, indexing strategies, and T-SQL querying.
- Azure Cosmos DB: Understanding of Azure Cosmos DB and its use cases for globally distributed, multi-model database workloads. Experience in designing and implementing Cosmos DB databases.
- Azure Stream Analytics: Knowledge of Azure Stream Analytics for real-time data processing and analytics. Ability to design and implement streaming data pipelines for real-time insights.
- Azure HDInsight: Experience in deploying and managing Apache Hadoop and Spark clusters using Azure HDInsight. Understanding of big data processing and analytics on Azure.
- Azure Data Explorer: Familiarity with Azure Data Explorer for real-time analytics on large volumes of data. Experience in ingesting, querying, and visualizing telemetry data and logs.
- Azure Blob Storage: Knowledge of Azure Blob Storage and its use cases for storing unstructured data such as images, videos, and log files. Experience in integrating Blob Storage with data engineering pipelines.
- Azure DevOps: Proficiency in using Azure DevOps for source code management, continuous integration, and deployment of data engineering solutions. Knowledge of pipelines, releases, and automation.
- Azure Monitoring and Diagnostics: Understanding of Azure monitoring and diagnostics tools for monitoring the performance, availability, and health of data engineering solutions deployed on Azure.
- Azure Security: Knowledge of Azure security services and features for securing data at rest and in transit. Understanding of role-based access control (RBAC), Azure Key Vault, and data encryption.
- Azure Resource Management: Experience in managing Azure resources using Azure Resource Manager (ARM) templates and PowerShell scripts. Knowledge of infrastructure as code (IaC) principles.
- Azure Cost Management: Understanding of Azure cost management tools and best practices for optimizing costs associated with data engineering solutions deployed on Azure.
- Bachelors degree in Computer Science, Information Technology, or a related field.
- 8 years of hands-on experience with Azure Data technologies.
- Proficient in SQL and T-SQL.
- Strong understanding of data warehousing concepts and best practices.
- Experience with big data technologies and frameworks.
- Excellent problem-solving and analytical skills.
- Effective communication and collaboration skills.
- Microsoft Azure certifications (e.g., DP-200, DP-201, DP-203) are a plus.
Similar jobs
Browse All Jobs
SD Worx
July 27, 2024
JPMorganChase
July 27, 2024
Data Scientist (Experimentation)- Senior Associate
London, England, United Kingdom
TWENTY ONE TALENTS
July 27, 2024
Data Engineer Expert ou Sénior Asap H/F
Saint-Denis-d’Orques, Pays de la Loire, France