Job description

  • Build data pipeline between source systems (RDS/SQL Server) to Snowflake on AWS
  • ETL data into AWS S3
  • ELT Data into Snowflake
  • Realtime data integration through CDC and AWS managed Kafka streams.
  • Embedded Tableau reporting
  • Data transformations using SQL, Python, PySpark, and DBT
  • Through understanding of data engineering principles including data lakes, and data warehouse
  • Data orchestration using Apache Airflow
  • Data Analysis skills
  • Best practices in data management, data quality and data stewardship were applied to ensure data integrity and reliability.
  • Optimize data pipelines for productivity and efficiency by considering factors such as scalability, reliability, and cost effectiveness.
  • Troubleshoot and resolve data infrastructure and pipelines in a timely manner.

Here’s What You’ll Do

  • Bachelor's or master’s degree in computer science, Engineering, or a related field.
  • Proven experience (6-10 years) as a Data Engineer
  • Proficiency in SQL for data extraction, transformation, and analysis across various database systems.
  • Strong programming skills in Python for data manipulation, scripting, and building data engineering solutions.
  • Experience in Tableau for data visualization, dashboard creation, and presenting insights.
  • Hands-on experience with PySpark for large-scale data processing and analysis.
  • Familiarity with Apache Airflow for workflow management and automation.
  • Practical knowledge of AWS services like S3, EC2, EMR, etc., for building data solutions.
  • Understanding and usage of Kafka for real-time data streaming and processing.
  • Specific expertise in Kafka Streams for advanced data stream processing.
  • Excellent problem-solving skills and the ability to work in a fast-paced environment.
  • Strong communication and collaboration skills with a proactive attitude towards learning and implementing new technologies.

Curiosity about Business Process, experience in optimizing and automating processes

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
February 28, 2024
The Walt Disney Company
February 28, 2024
February 28, 2024

Sr Data Analyst