Azure Data Engineer

ALGOTEQUE is an IT consultancy firm that helps startups, mid-sized and large corporations to create and deliver innovative technologies. Our team has a successful track record in designing, developing, implementing, and integrating software solutions (AI, ML, BI, Web, Automation) for Telecom, Energy, Bank, Insurance, Pharma, Automotive, Industry, e-commerce. We deliver our services both in fixed-price and time-and-materials models, helping our customers achieve their business and IT strategies.

Job Description

We are seeking a highly skilled and motivated Azure Data Engineer to join our dynamic team. The ideal candidate will be proficient in designing, developing, and maintaining scalable data pipelines and architectures using various Azure technologies. This role requires a deep understanding of cloud-based data solutions and the ability to deploy complex data processing systems.

Key Responsibilities

• Design, develop, and deploy scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks (ADB).
• Utilize PySpark within Azure Databricks environments to process large datasets efficiently.
• Manage and optimize Azure SQL databases for high performance and reliability.
• Develop data models and architectures for data ingestion, transformation, and aggregation.
• Implement data storage solutions using Azure Blob Storage, Data Lake, and other Azure storage technologies.
• Ensure data quality and integrity throughout the data lifecycle.
• Collaborate with data scientists and business analysts to gather requirements and deliver high-quality data solutions.
• Monitor, troubleshoot, and optimize data systems for performance issues.
• Maintain security standards and comply with data governance policies.
• Document technical processes, systems, and user guides for internal use.
  
  

Profile / Requirements

• Bachelor's degree in computer science, Engineering, or related field.
• Experience with Azure Data Factory, Azure Databricks, and PySpark.
• Strong knowledge of SQL and experience managing Azure SQL databases.
• Proficiency in developing and optimizing data pipelines and architectures.
• Experience with other Azure services like Cosmos DB, Azure Functions, and Azure Logic Apps is a plus.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.
• Valid EU/PL work permit.
  
  

Preferred Skills

• Certification in Microsoft Azure Data Engineering or similar credentials.
• Experience with machine learning workflows and models deployment in Azure.
• Knowledge of additional programming languages like Scala or Python.
  
  

AO2780