Job description

Responsibilities Design and implement data pipelines in Hadoop platform Understand business requirement and solution design to develop and implement solutions that adhere to big data architectural guidelines and address business requirements Fine-tuning of new and existing data pipelines Schedule and maintain data pipelines Drive optimization, testing and tooling to improve data quality Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, etc Build robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external users Work with data scientist and business analytics team to assist in data ingestion and data related technical issues Work with Group IT domains and outsourced vendors to deliver and implement solutions from requirement till post go-live. Ensure vendors to deliver solutions on-time, on-budget that fulfill business requirements and functionality, operational and scalableWe are committed to a safe and healthy environment for our employees & customers and will require all prospective employees to be fully vaccinated.Requirements Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent. Minimum 4 years of experience in data warehousing / distributed system such as Hadoop Experience with relational SQL and NoSQL DB Experience in building and optimizing ‘big data’ data pipelines, architectures and data sets Excellent experience in Scala or Python Experience in ETL and / or data wrangling tools for big data environment Ability to troubleshoot and optimize complex queries on the Spark platform

Not Specified

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
August 18, 2022

Data Engineer

August 18, 2022

ARG_JR Data Engineer

August 18, 2022

Data Engineer - Remote.