Lead Data Engineer

Job Purpose

At Great Eastern, we are building a data platform to realize the full value of our data. We seek an operational personnel to join our Group Data Division.

The primary purpose of this role encompass designing, building, and maintaining the systems and processes required to manage and analyses large volumes of data in an efficient, reliable, and secure manner. This involves working with other teams such as data scientists, data engineers, and data analysts to ensure that data is processed, transformed, and delivered in a timely manner to meet the needs of the business.

This is a hands-on role which requires technical expertise to deliver data and analytical services including data ingestion, transformation, storage, and reporting, and strives to continuously develop new and improved data engineering capabilities including data platform monitoring and operations, administration of the analytics platforms including Big Data Platform, Data Science platform and industry leading business intelligence and visualization products and leveraging a variety of DataOps, data warehousing and system administration methodologies and disciplines

The Job

• Drive optimization, testing and tooling to improve data quality and review solution design for data pipelines
• Design, create and maintain optimal data pipeline solutions to meet business, technical, user requirements.
• Ensure that proposed solutions are aligned and conformed to the big data architecture guidelines and roadmap.
• Support the data landscape and development best practices.
• Participate in proofs-of-concepts on platform innovation, and effectively transition and scale those concepts into production at scale.
• Contribute to platform enhancements, capacity management, performance monitoring, troubleshooting and resolution of technical issues.
• Identify, design, and implement process improvements including automating manual processes and optimizing data delivery.
• Technical ownership for data and data pipelines to ensure compliance with data standards, architectural standards, and achievement of business requirements.
• Streamline the design, development, and maintenance of applications based on data and data analytics.
• Ad hoc and system support to BI, analytics, and business applications.
• Participate in the management and automation of data pipeline, deployment, testing, release, security and monitoring processes.
• Participate in implementations, upgrades, updates, patches, enhancements, and customizations of the analytics systems in the data ecosystem.
• Assess and make optimal use of Big Data platforms especially Cloudera Data Platform CDP, AI/ML Platform Dataiku, MariaDB and Tableau.
• Improve the performance, security and organization of data assets.
• Identify and recommend hardware, software, and service upgrades where needed.
• Monitor and improve the performance, efficiency, effectiveness, and orchestration of our analytics systems across the data ecosystem.
• Provision access to analytics systems and data assets according to applicable policies and procedures.
• Participate in the creation of standards, procedures, design patterns and technical frameworks.
• Takes accountability in considering business and regulatory compliance risks and takes appropriate steps to mitigate the risks.
• Maintains awareness of industry trends on regulatory compliance, emerging threats and technologies in order to understand the risk and better safeguard the company.
• Highlights any potential concerns /risks and proactively shares best risk management practices.

Our Requirements

Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent with 8+ years of experience in distributed system such as Big Data Hadoop.

• Proficiency in Hadoop: Experience in Cloudera Data Platform CDP (Spark, HDFS, Hive, Sqoop, Airflow, Docker)
• Managing data infrastructure: Experience in monitoring of data platform infrastructure and incident handling.
• Knowledge of SQL: Experience in MariaDB or any other relational database management system.
• Experience with ETL: Experience in designing and implementing ETL processes, which includes data extraction, transformation, and loading from various sources into a centralized data warehouse.
• Programming skills: Experience in one or more programming languages such as Python, Java, or Scala, and be able to develop custom shell scripts and applications for data operation and monitoring
• Data warehousing: Good understanding of data warehousing concepts, including data architecture, dimensional modelling, and ETL processes.
• Analytical skills: They should be able to analyse complex data sets, identify patterns, and develop insights to support business decisions.
• Experience with DevOps: Experience in DevOps tools and environment (BitBucket, Jira, Bomboo)
• Experience in Agile Methodology
• Strong in User Requirements Gathering, Maintenance and Support
• Ability to analyze and understand complex problems
• Collaboration: Working closely with data analysts, data engineers, data scientists, and other stakeholders to understand their data needs and provide solutions to their problems. Should have good communication and collaboration skills.
• High level of integrity, takes accountability of work and good attitude over teamwork.
• Takes initiative to improve current state of things and adaptable to embrace new changes.

About Great Eastern

Established in 1908, Great Eastern places customers at the heart of everything we do. Our legacy extends beyond our products and services to our culture, which is defined by our core values and how we work. As champions of Integrity, Initiative and Involvement, our core values act as a compass, guiding and inspiring us to embrace the behaviours associated with each value, upholding our promise to our customers - to continue doing our best for them in a sustainable manner.

We work collaboratively with our stakeholders to look for candidates who exhibit or have the potential to embrace our core values and associated behaviours, as these are the key traits that we expect from our employees as they develop their careers with us.

We embrace inclusivity, giving all employees an equal opportunity to shine and play their role in exploring possibilities to deliver innovative insurance solutions.

Since 2018, Great Eastern has been a signatory to the United Nations (UN) Principles of Sustainable Insurance. Our sustainability approach around environmental, social, and governance (ESG) considerations play a key role in every business decision we make. We are committed to being a sustainability-driven company to achieve a low-carbon economy by managing the environmental footprint of our operations and incorporating ESG considerations in our investment portfolios; improving people’s lives by actively helping customers live healthier, better and longer; and drive responsible business practices through material ESG risk management.

To all recruitment agencies: Great Eastern does not accept unsolicited agency resumes. Please do not forward resumes to our email or our employees. We will not be responsible for any fees related to unsolicited resumes.