Senior Data Engineer (Databricks, Scala)

Description

Currently, we are looking for Senior Data Engineer with Databricks and Scala expertise to make our team even stronger.

Our client is a publicly traded German online fashion and beauty retailer active across Europe.

Responsibilities

• Develop, monitor, and operate the most used and most critical curated data pipeline - Sales Order Data (incl. Post-order information, e.g. shipment, return, payment). This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases
• Consulting with analysts, data scientists, and product managers to build and continuously improve Single Source of Truth KPI for business steering such as the central Profit Contribution measurement
• Leverage and improve a cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala
  
  

Requirements

• Expertise in Apache Spark along with Spark streaming & Spark SQL
• Experience with Databricks and delta-lake
• Fluency in Scala programming language
• Good understanding & experience with CI/CD
• Rich working experience with Github
• Fluency working with AWS landscape
• Ability to build Apache Airflow pipelines
• Upper-Intermediate level of English, both spoken and written (B2+)
  
  

Nice to have

• Presto
• Exasol
• Starburst
• Superset
  
  

We offer

• Competitive compensation depending on experience and skills
• Variety of projects within one company
• Being a part of a project following engineering excellence standards
• Individual career path and professional growth opportunities
• Internal events and communities
• Flexible work hours