Senior Data Engineer (Hadoop, Hive, Spark, Azure Data Factory)
One of my globally leading clients is urgently looking for a Senior Data Engineer to join them on an 18-month contract. The client is based in Amsterdam and as of now the team is working from home, but should things change flexibility is required.
As part of our client's Back Office team within Managed Sourcing, you will coordinate and plan for the onboarding of many sources to our enterprise-wide data distribution platform (DIAL). The onboarding is done through the description of metadata in business and technical templates in close collaboration with the appropriate stakeholders in the organization. These templates are then delivered to the Back Office, where the technical interface between DIAL and the providing application is setup and the interface is deployed to DIAL through the DTAP process. Additionally, the metadata is loaded into our metadata repository.
The Team Objective:
- Data is onboarded to DIAL and made available through Data Marketplace (DMP).
- We conform to the overall planning and facilitate the onboarding in a transparent and timely matter.
- Team KPI's, load and output are constantly being measured and made available for discussion.
- Application Owners feel supported and know what is expected of them.
The ideal Senior Data Engineer will have the following responsibilities:
- Orchestrate and coordinate the technical onboarding of all applications to DIAL following a structured process.
- Guide a team of developers who deploy the technical interfaces and perform the steps necessary to load the metadata in the repository.
- Guide the team in using the latest enablers, since many parts of this process are getting automated the Back Office scope is slowly changing.
- Align with the appropriate stakeholders (mostly Application Owners) in the organisation to make sure they succeed in onboarding their data to DIAL.
- Make sure the team capacity and output are properly measured, made transparent and connected to the demand.
- Align the onboarding planning with the overall Managed Sourcing planning since you will be the last step in the process.
The ideal Senior Data Engineer must have the following skills:
- Enough technical knowledge from big data technology (Hadoop, Hive, Spark, Azure Data Factory) to be able to guide the developer team and to understand and optimise their work.
- Stakeholder management skills
- Project management (planning) skills
- Analytical and communication skills
- Supportive mindset
- Familiar with generic data management topics (both IT and Business)
- Excellent Dutch & English communication skills
- Able to maintain focus in a complex and ever-changing environment.
Please get in touch ASAP for further details