Lead Data Engineer

job description:

Hiscox is seeking a strong and dynamic “hands on” Lead Data Engineer to join our Re & ILS Technology Team. You will help us to create a professional data engineering practice, architecting and building a greenfield cloud based data analytics platform to support our new data strategy.

We are looking for an enthusiastic and self-motivated individual with a thirst for cloud agnostic tech, data processing/analytics, machine learning and good data governance knowhow to help grow and lead the overall team in this area.

The successful candidate will help architect and build new data solutions and initial PoCs using SaaS and PaaS-based services on Azure. While Microsoft PaaS data services are often our starting point, knowledge and experience of non-Microsoft PaaS services is welcomed and encouraged, including hosted services on other Cloud providers. The ability to critique and assess whole-of-industry options (both functionally and non-functionally) and communicate options is imperative.

Our role requires someone with a strong understanding of how to set up the appropriate tools and technologies in the cloud to securely process big and small, unstructured and structured data – from ingestion, cleansing, transformation, enrichment, outlier/anomaly detection, auto matching/classification and finally to the delivery of accurate data for insights, analysis and reporting.

Knowing how to deploy fast R&D solutions and robust operational/production systems is key and having previous experience of implementing secure, scalable and automated data pipelines and self-service business intelligence solutions is a must.

We need someone who can get stuff done and deliver value incrementally to the business – who completely understands what agility really is and always looks to keep things simple, understands best practices in the cloud, the importance of DevOps and ensures that we can easily build, deploy and monitor our solutions when they go into production – keeping an eye on infrastructure costs.

You will be working alongside our existing tech team, Data Architect, and our Data Science team. You’ll need excellent communication skills and the ability to mentor and educate others in the various data processing technologies that we adopt and implement.

As Lead Data Engineer your role carries the responsibility and ownership of making technology choices and influencing the overall design and direction of Re & ILS’s digital transformation in relation to the way it processes and utilises its various data assets. In return we will ensure you can fully self-organise, have autonomy and flexibility in your decision making.

You will have a number of years’ experience in Data engineering and prior experience growing and leading a team of data engineering experts.

The role is ideally suited to someone interested in working with the latest cloud technologies in a dynamic environment, in the face of high performance, complex data wrangling and high variety of data asset challenges.

Key areas of work:

• Working with senior stakeholders to understand high value business problems that can be solved through the application of data processing and analytical systems
• Helping to architect, setup and support a new cloud-based analytics platform for the business
• Helping to create and manage a specialist Data Engineering practice for the Re & ILS’s business
• Ensuring your team supports the Data Science Team by building data pipelines for specific insights and provides them with an environment for their R&D and experimentation
• Providing robust, secure and scalable production data analytics solutions
• Understand business requirements and refine into development tasks and estimate their complexity
• Research, evaluate and adopt new technologies with a right tool for the job mentality
• Prototype, fail rapidly and iterate
• Focus on both speed of delivery and quality, with suitable pragmatism – ensuring your solutions are always “appropriate” and not overly complex or over-engineered
• Quick progression of projects from PoC to post-production stage
• Communication and presentation of ideas to colleagues in all parts of the wider business
• Recruitment, Mentoring & code reviews for the Data Engineering Team

What should excite you about the role:

• The business:
  • Leading brand across the (re)insurance space
• The working environment:
  • Great working space in the heart of the city
  • Buy-in from the senior management to implement data analytics
  • A collaborative IT department that is using the latest technology
  • Global community of analytical professionals
• The role:
  • Highly visible role with access to the business leadership
  • Continued professional development and exposure to business
  • Competitive compensation that rewards high achievers

Required skills:

• Passion for and experience of building data analytical platforms to deliver business value.
• Excellent attention to detail and accuracy is a must.
• Excellent troubleshooting skills and the ability to quickly identify the source of any data errors or performance issues and solve these.
  • Track record of taking initiative and delivering projects end-to-end; clear evidence of being self-driven and motivated.
  • Immense curiosity, high energy and desire to go the extra mile to make a difference.
  • Having an awareness of trends in the data engineering space.
  • Leading and nurturing the team.

• Ability to setup secure, automated and scalable data platforms / technologies within Microsoft Azure.
• Good awareness of data services available across the market with an ability to rationally compare and communicate options, considering both functional and non-functional factors.
• Good working knowledge of various database types and when to use them, i.e. SQL, Columnar, Graph, Key-Value, and Document.
• Solid experience working with ELT/ETL tools like Data Factory, Matillion and SSIS.
• Solid experience working with Azure Storage and Azure Data Lake.
• Good understanding of different types of data (i.e. structured, semi-structured and unstructured data) and the considerations that should be made in how they are stored.
• Solid experience of working with and implementing BI solutions on platforms like Tableau and Power BI.
• Solid understanding on how to incorporate data lineage, discovery and governance within production solutions, with experience in tools such as Purview, Informatica, or Apache Atlas.
• Solid experience of data management and validation techniques.
• Excellent knowledge of database design patterns – including OLTP, OLAP and overall schema design
• Solid experience of traditional data warehousing (Kimball/Inmon) and modern warehousing techniques and services (e.g. Synapse Analytics, Snowflake).
• Excellent working knowledge of T-SQL.

• Experience of ensuring data is properly backed up, resilient, and understand the importance of backup testing.

Desirable Skills / Experience:

• Strong knowledge of data streaming concepts, technologies and patterns.
• Understanding of Delta Lake concepts.
• Ability to code in at least two of these languages: .NET (C#), Python, Scala and R.
• Experience working with NoSQL databases or CosmosDB.
• Good understanding around Machine Learning – ideally anomaly detection, classification, predictive analytics, NLP & recommendation (when working alongside our Data Scientists).
• Good knowledge around Continuous Integration, auto-build and auto-deployment technologies (e.g. Azure DevOps pipelines).
• Good knowledge of infrastructure-as-a-code scripting like Terraform and ARM
• Containerisation experience with Docker in AKS or Azure CI.
• Hands-on experience in the Insurance Sector, Financial Services or Data-Driven Start-up
• Consulting experience.
• A degree in applicable area.

person specification: