Cloud Data Engineer

Company:
Location: St. Louis, MO

*** Mention DataYoshi when applying ***

Cloud Data Engineer, St. Louis, MO


Responsibilities

  • Develop optimal extract, load and transform pipelines from a wide variety of data sources using Python and Spark
  • Utilize GCP ‘big data’ technologies like Cloud Dataproc and BigQuery
  • Schedule and orchestrate solutions using GCP Cloud Composer
  • Create real-time data streaming pipelines using GCP App Engine and Dataflow
  • Perform data exploration in support of new projects and to troubleshoot issues
  • Work closely with data analysts to facilitate data acquisition and enable efficient analysis
  • Adapt existing batch and real time data pipelines to accommodate new business requirements
  • Develop and deploy code with exception handling, logging and monitoring
  • Ensure data quality throughout the entire data processing pipeline
  • Perform unit testing, integration testing, and assist with user acceptance testing
  • Be proactive in clarifying requirements when needed to ensure accurate results

EXPERIENCE


  • Bachelor’s Degree in computer science or equivalent experience required
  • 3+ years hands on experience with Python
  • 3+ years hands on SQL and relational database experience
  • 2+ years hands on experience in a public cloud development environment (GCP preferred)
  • Experience setting standards for design, documentation, testing, and code quality
  • Proficient understanding of distributed computing principles
  • Experience with messaging systems, such as Kafka and GCP PubSub
  • Ability to conduct code reviews and provide constructive feedback to peer team members
  • Excellent Communication Skills

*** Mention DataYoshi when applying ***

Offers you may like...

  • Artemis Consulting Inc

    Senior AWS Cloud Data Engineer
    Washington, DC 20001
  • Ankix

    Cloud Data Engineer (m/f)
    Lisboa
  • Parallel Consulting

    Cloud Data Engineer
    Wien, W
  • Anson McCade

    Cloud Data Engineer (AWS Serverless)
    Sydney NSW 2000
  • ANALYGENCE, Inc.

    Cloud Data Engineer/Information Engineer
    Reston, VA