Energy Jobline

Senior Data Engineer IRC203967

Job description


Experian is the world's leading global information services company, unlocking the power of data to create more opportunities for consumers, businesses, and society. We are thrilled to share that FORTUNE has named Experian one of the 100 Best Companies to work for. In addition, for the last five years we've been name in the 100 World's Most Innovative Companies by Forbes Magazine.

Experian Marketing Service's mission is to accelerate client success through enabling ecosystems, partnerships, and technology solutions. We help brands put people at the heart of their business and have meaningful interactions with their customers.

Founded in 2010, Experian contains the most robust global cross-device digital identity graph on the market. This Graph enables marketers to maximize their digital marketing investment for years to come. As a leading provider of consumer data analytics and targeting solutions, Experian Marketing Services has a rapidly growing need to continue expanding our strategy on identity data and services to meet market demand.


  • DBT core experience is must
  • Container Images: Docker
  • Container Registry: TBD
  • Compute Orchestrator: Kubernetes (Managed version EKS)
  • Data Pipeline Orchestrator: Airflow | Kestra
  • Data Transformation Frameworks: Spark & DBT
  • Programing Language (Data Pipeline): Scala & sbt
  • Configuration Management: AWS App Config | AWS Parameter Store
  • Code Repository: Github / git
  • Object Storage: S3
  • Table Metastore: AWS Glue
  • Open Table Format: Iceberg
  • DBT Compute Engine: AWS Athena (possibility of also covering Snowflake)
  • Visualization: Looker | Tableau

Job Responsibilities

  • Development of a File Ingestion Framework (Spark/Scala)
  • Read files from object storage (landing zone).
  • Convert file to open table format Iceberg.
  • Perform some column label categorization.
  • Schema Inference, pushed to central repository and data services.
  • Development of Transformation Application (Spark/Scala)
  • Read Iceberg tables (raw/refined zone).
  • Translate legacy application code into next-gen tech stack (see above).
  • Write datasets back into proper lake zone.
  • Development of dbt data models/macros, setup standards of development for other less-technical teams to continue expanding the code-library following proper procedures.
  • Work collaboratively with Software Engineers around the Orchestration component of our applications.
  • Development of Export Framework
  • Easily export files from our lake into agreed upon channels
  • IAM Controls.

What We Offer

Why work at GlobalLogic

Our goal is to build an inclusive positive culture where everyone can feel comfortable being themselves, empowering our people to create their own high standards and therefore more value. We work together to promote fairness while recognising, valuing and embracing differences - providing a transparent support structure and generous training budget to help our people develop skills to progress their career. Our region also supports a hybrid model which can flex across a wide spectrum of working options determined by our business, customer and individual needs.

You'll benefit from a comprehensive health and wellness plan, private healthcare (clinical and mental wellbeing), and discounted gym memberships. We offer a fantastic benefits package including a competitive pension scheme and recognition schemes through bonus/reward initiatives. Colleagues are entitled to an annual volunteering day - so you can take time to support a cause close to your heart. We also love to stay social at our trips to the zoo, quiz nights, sports events, theatre trips and much more.

We are an equal opportunities employer. It is our policy to promote an environment free from discrimination, harassment and victimisation. We are proud signatories to the Tech Talent Charter which was created because solving the diversity problem in tech requires a collective effort across companies and sectors -

About GlobalLogic

GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world.

By integrating experience design, complex engineering, and data expertise-we help our clients imagine what's possible, and accelerate their transition into tomorrow's digital businesses.

Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries.

GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501) which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.