Senior data engineer

Job description


Innovation & Engineering

Job Family Group:

IT&S Group

Job Summary:

As a bp Data Engineer you will be developing and maintaining data platforms and infrastructure, along with writing, deploying and maintaining software to build, integrate, manage, maintain, and quality-assure data. You will be part of bp’s Data & Analytics Platform organisation, the group responsible for the platforms and services that underpin bp’s data supply chain.

For this role specifically within Knowledge and Data Stores, you will be involved in designing, implementing and delivering the necessary platforms to enable master and reference data management, data modelling, searching and indexing, and data transformations and movement for data in bp’s enterprise systems. This role is key in ensuring availability and reliability of quality data that bp’s enterprise organisation can use to detect, anticipate and prevent issues, provide insights to improve our operations, and have timely access to data that will allow our platform engineers to build the necessary automation for operational efficiency, self-healing, etc.

Job Description:

Key accountabilities

Design, select, and implement data and knowledge management solutions within bp. Implement, and guide direction of platforms for master and reference data management, knowledge management, searching and indexing, and oil and gas data frameworks. Work with business developers and users to provide experienced project guidance and assure alignment to processes. Work with third party vendors and open source collaborations to assure bp needs are met within selected systems.

Maintain toolsets in areas such as master data management, search and indexing, and data modelling, and facilitate effective use by business users, developers and architects across the corporation. Provide architectural and business process guidance to platform engineers and business users. Support on these platforms the creation of data pipelines for Azure and AWS data platforms and services, provisioning of data nodes and assuring telemetry for performance and utilisation analytics.

Collaboration with enterprise platform teams to apply these data platforms and existing data products, ingestion patterns, or automations to avoid tailored development while contributing to the enhancement and creation of these shared assets when gaps are identified

Write, deploy and maintain software to build, integrate, manage, maintain, and quality-assure data, and responsible for deploying secure and well-tested software that meets privacy and compliance requirements; develops, maintains and improves CI / CD pipeline.


  • Deep and hands-on experience designing, planning, implementing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environments.
  • Experience with data management patterns and standard practices and Agile development.
  • Development experience in one or more object-oriented programming languages (e.g. Python, Go, Java, C++)
  • BS degree in computer science or related field

Desirable Criteria

  • Data Manipulation: debug and maintain the end-to-end data engineering lifecycle of the data products; design and implementation of the end-to-end data stack, including designing complex data systems, e.g. interoperability across cloud platforms; experience on various types of data (streaming, structured and un-structured) is a plus.
  • Software Engineering: hands-on experience with SQL and NoSQL database fundamentals, query structures and design standard practices, including scalability, readability, and reliability; you are proficient in at least one object-oriented programming language, e.g. Python [specifically data manipulation packages - Pandas, seaborn, matplotlib], Apache Spark or Scala
  • Scalability, Reliability, Maintenance: proven experience in building scalable and re-usable systems; knowledge and experience in automating operations and building for long-term productivity over short-term speed/gains, and fulfill those opportunities to improve products or services.
  • Cloud Engineering– Recent experience applying data analytics offerings and services from Azure and AWS

Why join us

At bp, we support our people to learn and grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. We are committed to fostering an inclusive environment in which everyone is respected and treated fairly.

There are many aspects of our employees’ lives that are important, so we offer benefits to enable your work to fit with your life. These benefits can include flexible working options, a generous paid parental leave policy, and excellent retirement benefits, among others!

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.

Travel Requirement

Negligible travel should be expected with this role

Relocation Assistance:

This role is not eligible for relocation

Remote Type:

This position is a hybrid of office/remote working


Legal Disclaimer:

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, neurodiversity/neurocognitive functioning, veteran status or disability status. Individuals with disabilities may request a reasonable accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an accommodation related to the recruitment process, please contact us to request accommodations.

If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.