Data Engineer

Sembcorp is embarking on a major transformation journey that focused on improving value, deepen market presence, and capture product innovation and growth. At the core of this transformation is the Sembcorp Digital business aiming to strengthen its leadership position through the modernization and innovation of its technology capability. We are currently looking for a Senior Data Engineer with experience in Microsoft Azure Cloud, Amazon & Google. In this role, you will be responsible for building real-time/batch data ingestion pipelines integrating different data sources like RDBMS, APIs, plant legacy systems, historian systems, storage systems etc.

Responsibilities

• Design, develop, and launch extremely efficient and reliable data pipelines.
• Solve issues in the existing data pipelines and build their successors.
• Build modular pipeline to construct features and modelling tables.
• Maintain data warehouse architecture and relational databases.
• Monitor incidents by performing root cause analysis and implement the appropriate action.
• Create, document and monitor highly readable code.

Requirements

• Masters/Bachelor’s degree in Computer Science or any other related field
• Minimum 4 years’ experience in designing, building and operationalizing medium to large scale data integration (structured & unstructured) projects with Azure Data Lake, Data Warehouse, BLOB Storage, RDBMS using Azure integration services.
• Minimum 1 project implementation of real-time data integration & processing.
• Hands-on Experience in following big data technologies in the Microsoft Azure cloud: Data Lake Store, Data Warehouse, Data Catalog, Data bricks, Data Lake Analytics, Stream Analytics, Event Hub, IoT Hub, Data Factory, and Logic Apps
• Hands-on experience in CI-CD implementation with Azure DevOps.
• Hands-on experience in scripting using JavaScript, PowerShell, Python is must. Knowledge of node JS, react, angular JS will be an added advantage.
• Hands-on experience in Programming languages Python & .Net
• Hands-on experience in SQL/T-SQL and query optimization with SQL DB and DW.
• Minimum 2 years of designing and developing reports and Dashboards using Power BI/Tableau/Qlik/SSRS

Added Advantage

• Data engineering with Spark framework.
• Real-time data ingestion using Spark streaming
• Data integration with other cloud services like AWS SQS, AWS S3, AWS Glue and Google cloud platform etc.
• Prior knowledge in relational SQL and NoSQL databases such as Neo4j and MongoDB, Python, UNIX or statistical software.
• Certified in Microsoft technologies

Preferred Skills & Characteristics

Consistently display dynamic independent work habits, goal oriented, passionate in growth mindsets, possess a ‘can do’ attitude, and self-motivated professional. Self-driven and proactive in keeping up with new technologies and programming.