We are looking for a talented Data Engineer to join our growing team. You will be responsible for expanding our data collection as well as optimizing data pipelines for cross-functional teams. Ideally, you are an experienced engineer who enjoys optimizing data pipeline and building them from scratch. You will support business analysts, software engineers, and data scientists and ensure optimal on-time delivery of ongoing projects. You will be a self-starter and thrive in supporting the data needs of multiple teams, systems, and products. Experience in e-commerce industry will be a plus.
Roles & Responsibilities:
Build both batch and streaming data pipeline for optimal extraction, transformation, and loading of data from a wide variety of data sources using big data technologies
Deliver data projects to support customer acquisition, operational efficiency, campaign and other key business performance metrics
Monitor & manage data pipelines, ensuring accuracy and stability
Diagnose, analyse and provide solutions to issues in scripts, reports, tools, data, etc.
Work with stakeholders and sort out data-related issues
Identify, design and implement improvements, e.g. automating manual processes, optimizing data delivery, re-designing architecture for greater scalability, etc.
Provide on-call support on a roster basis
Bachelor degree in Computer Science, Electrical Engineering or similar
5+ years of relevant working experience
Familiar with Hadoop big data platform architecture: HDFS/HBase/Hive/MapReduce, proficient in Hive SQL, understand real-time computing, and have previous experience in using open source real-time streaming engines such as Flink and Spark Streaming
Familiar with dimensional data modelling, and data warehouse concepts and methodologies as well as distributed processing systems
Experience with programming and understanding of basic algorithms