Technopals Pte Ltd

Intermediate Data Engineer

Job description

Mandatory Skills

We are seeking a skilled and motivated Intermediate Data Engineer to join our dynamic team. The successful candidate will play a crucial role in designing, developing, and maintaining our data infrastructure, ensuring efficient processing and analysis of large datasets. As an Intermediate Data Engineer, you will collaborate with cross-functional teams to implement data solutions that align with our business objectives.

Responsibilities

  • Data Pipeline Development: Design, implement, and maintain scalable data pipelines for the extraction, transformation, and loading of large datasets.
  • Data Modelling: Develop and optimize data models to support analytical and reporting requirements, ensuring data accuracy and reliability.
  • Data Integration: Integrate diverse data sources and formats to enable seamless data flow across systems.
  • Data Quality Assurance: Implement and monitor data quality checks to ensure the integrity of the data throughout the entire data lifecycle.
  • Collaboration: Work closely with business analysts, colleagues and other stakeholders to understand data requirements and deliver effective solutions.
  • Performance Optimization: Identify and address performance bottlenecks in data processing and storage systems.
  • Documentation: Document data engineering processes, data models, and configurations for knowledge sharing and future reference.
  • Adherence to Best Practices: Stay current with industry trends and best practices in data engineering within company and ensure the adoption of these practices within the team.

Requirements

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Minimum of 5 years of proven experience as a Data Engineer or similar role.
  • Strong proficiency in programming languages such as Spark SQL and Python in a HIVE environment.
  • Familiarity with cloud platforms such as AWS and Big Data infrastructure, Jira, Bitbucket, etc.
  • Experience in CI/CD pipeline and working in an Agile environment.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills.
  • Knowledge of data warehousing concepts and technologies.
  • Candidates with experience in the Banking Industry will have an added advantage.
  • Ability to communicate effectively with multiple levels of management and summarize complex technical requests succinctly.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.