In this role, you will have the opportunity to shape the data strategy and structure of our client's sports data portfolio. As the Data Engineer, you will be responsible for designing, developing, and maintaining comprehensive data lake and warehouse solutions. This opportunity will allow you to work with one of the world's largest and leading news corporations.Key Responsibilities
- Building different types of data lake and warehousing layers based on specific use cases.
- Building scalable data infrastructure and understanding distributed systems concepts from a data storage and compute perspective.
- Utilizing expertise in SQL and having a strong understanding of ETL (Extract-Transform-Load) and data modeling.
- Ensuring the accuracy and availability of data to customers and understanding how technical decisions can impact the business's analytics and reporting.
- Interfacing with other technology teams to extract, transform and load data from a wide variety of data sources.
- Developing scalable engineering solutions and building data solutions that drive real impact within the organization.
- Ensure quality by performing root cause analysis and troubleshooting of defects.
- Bachelor's degree in Computer Science, Software Engineering, or a related field.
- 3+ years of experience in Data Engineering, with a strong focus on batch and stream data processing using distributed computing systems like Spark.
- Proficiency in either pySpark or Scala to handle large-volume data processing.
- Proficiency in programming languages like Python or Java.
- Familiarity with AWS and Databricks.
- Experience with Agile software development methodologies.