Data engineer

Overview: We are seeking a Data Engineer to join our Quantification Analytics team. The ideal candidate will possess strong expertise in data engineering and data warehousing, driving forward our Quantum modelling efforts. There are vast amounts of data available internally and externally to model for the purposes of Quantification and this role will be key in structuring the data for reporting and analytics.

Key Responsibilities

• Data Pipeline Development: Design, develop, and maintain robust data pipelines and ETL processes using Azure Data Factory, Azure Databricks, and other Azure data services.
• Database Management: Implement and manage scalable data storage solutions, including Azure SQL Database, Azure Synapse Analytics, and Azure Data Lake.
• Data Integration: Integrate data from various sources, ensuring data consistency, quality, and security across different systems.
• Performance Optimisation: Monitor and optimise data processes for performance, scalability, and reliability. Troubleshoot and resolve any data-related issues.
• Collaboration: Work closely with data scientists, data analysts, and other stakeholders to understand data needs and provide technical solutions. Ensure seamless data flow and accessibility for analytical and reporting purposes.
• Data Governance: Implement and maintain data governance practices, ensuring data privacy, security, and compliance with relevant regulations.
• Documentation: Document data processes, architectures, and workflows to ensure knowledge sharing and maintainability.
  
  

Qualifications

• Education: Bachelor s or Master s degree in Computer Science, Information Technology, Data Science, or a related field.
• Experience: Minimum of 5-7 years of experience in data engineering/data warehousing.
• Technical Skills: o Proficiency in SQL and experience with relational databases (Azure SQL Database, SQL Server).
• Experience with big data technologies and frameworks (Azure Synapse Analytics, Azure Data Lake, Hadoop, Spark).
• Strong programming skills in Python, Java, or Scala.
• Familiarity with data integration tools and platforms (Azure Data Factory, Apache NiFi).
• Knowledge of cloud computing and data services, particularly Azure.
• Analytical Skills: Strong problem-solving skills with the ability to analyse complex data systems and processes.
• Communication Skills: Excellent verbal and written communication skills, with the ability to collaborate effectively with both technical and non-technical stakeholders.
• Attention to Detail: High attention to detail and a commitment to data quality and accuracy.
• Team Player: Ability to work collaboratively in a team environment and manage multiple projects simultaneously.
• Meal Allowance: R$ 1,210.00 (Flexible via Caju: supermarket or restaurant)
• Health Insurance: SulAmérica with 100% coverage for the employee (not extendable to family)
• Life Insurance
• Gympass
• Transportation Voucher
• Bonus: Discretionary, based on company and employee performance.