- Degree in information technology / mathematics / engineering / actuarial science or related discipline.
- 8+ Years in a technical data role, preferably in a formal data, data warehouse or business intelligence environment.
The Lead Data Engineer is responsible for creating business value by leading a team that apply data engineering and data management disciplines to design and build data solutions that enable data driven decision support, in order to optimize business decisions and processes.
- Identify data sources that can add value to decision making.
- Work with source system owners and analysts to understand source data, e.g. data profiling, definition and mapping.
- Design and implement efficient data loads, using traditional structured data ETL techniques.
- Design and implement real time and near real time data load solutions, using technologies like data streaming.
- Design and implement unstructured data loads, e.g. text speech, images and video.
- Design and implement load monitoring tools and procedures and perform continuous monitoring and optimising of loads.
- Work with analysts and architect to design and implement effective and efficient data warehouse data models using appropriate modelling techniques.
- Design and implement data pipelines for ad hoc, unstructured and other data models.
- Design and implement appropriate aggregation data structures that enhance usability of data, e.g. multi- dimensional OLAP structures, summary tables etc.
- Design, implement and maintain appropriate indexing on tables to enhance speed of access.
- Design and implement data models that support automated decision making and/or further analytics.
- Continuously search for data elements from other sources to enhance existing data objects to supplement/ enhance context.
- Design and implement data monitoring solutions and procedures and continuously monitor and maintain integrity of existing environment, troubleshoot technical and data issues and make appropriate changes where required.
- Design and implement meta-data solutions that assist with understanding and managing data.
- Work together with business owners, analysts and IT to manage changes to data in the organization and maintain good data governance.
- Provide technical and data related support to source system teams and external parties with whom we exchange data.
- Manage data growth and usage by implementing effective strategies, e.g. archiving and indexing.
- Manage systems, technology and tools that enable data management and analytics and liaise with IT infrastructure and IT Operations regarding system and infrastructure management.
Personal, People and Functional leadership
- Take ownership of own work by delivering high quality work on time.
- Show initiative and be pre-active in finding opportunities to improve data and/or processes.
- Take ownership of own career development by continuously improving skills, knowledge and the application thereof in designing and implementing solutions.
- Positive engagement in team activities and actively contribute ideas to improve team dynamics and performance.
- Cross functional data and team knowledge gathering and sharing.
- Responsible for team activities, team dynamics and performance.
- Manage project and task delivery of team as well as quality control there off.
- Functional Competencies
- Data analysis
- Data visualisation
- Data modelling
- Microsoft business intelligence data technologies (SSIS, SSAS, SQL Server)
- Data warehouse concepts and best practices