Data Engineer

Location: Ang Mo Kio

*** Mention DataYoshi when applying ***

Key Responsibilities:

  • Participate in the design, development, and testing of highly scalable, expandable and maintainable big data platform that ingests and stores large volumes of structured/unstructured data from disparate sources and with different latencies (e.g., on-demand, batch, real-time, near-real-time);
  • Design, develop, document, manage and improve the data schema and data architecture of the big data platform;
  • Design, develop, test, manage and support the SLAs of batch and streaming data pipelines, ETL/ELT processes, data queries for the big data platform;
  • Design and implement appropriate data stores like databases, data warehouse and/or data lake to store data efficiently, keep the data up-to-date and serviceable for external systems or end-users to query;
  • Work with data architects, data analysts, data scientists and other internal stakeholders with regards to soliciting data requirements to support machine learning modelling works, preparing the data, updating the data pipelines and data serving infrastructure of the big data platform as required;
  • Troubleshoot system and software issues related to the big data platform; and
  • Other duties/responsibilities as assigned.

Job Requirements:

  • Degree in computer science/engineering, software engineering, knowledge engineering, information systems or equivalent with specialisation in data analysis, data management or data engineering;
  • Recognised professional or industrial certifications in relevant technologies or competencies;
  • The candidate should have at least 2 years of technical experience in Information Technology with at least 1 years in Big Data, Data Lake, Data Warehousing or Business Intelligence technology with knowledge of Data Analytics and AI technologies;
  • Fresh graduates and candidates with less experience but with strong interest in this field are welcomed to apply and may be considered for this position or other suitable positions in our organization;
  • Broad knowledge of various aspects of Big Data with good understanding and hands-on experience in Hadoop-based technologies such as Hive, HBase, Spark, Kafka, Flume, Nifi etc;
  • Good knowledge in relational, NoSQL database technologies such as PostgreSQL, MongoDB, Elasticsearch etc;
  • Good knowledge in scripting, programming languages such as Java, Python, Scala, SQL on Linux/Windows platforms;
  • Experience in master data management, data governance, data lifecycle management etc;
  • Understand and implement solutions that ensure adherence to data security and data governance standards/requirements;
  • Drive/execute data quality assurance and data privacy practices;
  • System-to-system integration development using API, messaging and related technologies;
  • Experience in data visualization and enterprise reporting software will be an added advantage;
  • Experience in designing ETL/BI solutions will be an added advantage;
  • Experience in DevOps, MLOps will be an added advantage;
  • Experience in operational support in delivering Big Data solutions will be an added advantage;
  • Familiar with Linux/UNIX system administration will be an added advantage;
  • Familiar with Azure Cloud or other cloud technologies will be an added advantage;
  • Familiar with open-source software technologies in the Big Data solutions space will be an added advantage;

*** Mention DataYoshi when applying ***

Offers you may like...

  • Glassdoor

    Data Engineer (Remote)
    Washington, DC
  • Homethrive

    Data Engineer
  • YouGov

    Data Engineer
    New York, NY
  • drc

    Senior Full Stack Developer / Data Engineer - REMO...
    Maple Grove, MN
  • Relatus

    Senior Data Engineer