Senior Data Engineer - Spark/Scala (Remote)

Company:
Location: Remote

*** Mention DataYoshi when applying ***

About MasterControl:

MasterControl Inc. is a leading provider of cloud-based quality and compliance software for life sciences and other regulated industries. Our mission is the same as that of our customers to bring life-changing products to more people sooner. The MasterControl Platform helps organizations digitize, automate and connect quality and compliance processes across the regulated product development life cycle. Over 1,000 companies worldwide rely on MasterControl solutions to achieve new levels of operational excellence across product development, clinical trials, regulatory affairs, quality management, supply chain, manufacturing and postmarket surveillance. For more information, visit www.mastercontrol.com.


Summary

At MasterControl we are building our next generation data platform that will leverage AI/ML techniques to help redefine how our customers bring lifesaving and lifechanging products to market. To enable this, we need your help building our Data Pipeline and Data Lake.

We are looking for a Senior Stream Processing Data Engineer using Apache Spark Structured Streaming, Scala, and AWS EMR.

If you have experience in other stream processing frameworks, such as Flink, Apache Beam, Kafka Streams, etc., and are willing to learn Spark, we are interested in talking to you!


Responsibilities

  • Transforming structured, semi-structured, and unstructured data for use by BI/ML/AI processes
  • Build scalable data processing pipelines in Spark
  • Write comprehensive unit and integration tests for all data processing
  • Collaborate with DevOps/BI/AI/ML/Product Teams
  • Analysis of transformation requirements
  • Other assigned duties


Required Skills

  • Competent with Scala, Spark, and the Spark Dataframe/Dataset API
  • Scala (preferred), Java (meh, okay), Python (super cool, but let's elevate to Scala)
  • ELT/ETL Best Practices
  • Big Data mindset and experience
  • Investigate and research new technologies
  • Competent in basic SQL
  • Linux Shell/GNU
  • Mentor team members, especially the stupid manager
  • Meet multiple, challenging deadlines while communicating expectations clearly
  • Must be able to work well with people
  • Ability to communicate effectively in writing, in person, over the telephone and in e-mail


Desired Skills

  • AWS
  • Terraform or Cloud Formation
  • Kafka
  • Hudi/Delta Lake
  • Airflow


Why Work Here?

#WhyWorkAnywhereElse?

MasterControl is a place where Exceptional Teams come together to do their best work. In fact, hiring Exceptional Teams is a core value of ours. MasterControl employees are surrounded by intelligent, motivated, and collaborative individuals. We like to call it #TheBestTeamOnThePlanet.

We work hard to develop and challenge our employees' skillsets, recognize their contributions, encourage professional development, and offer a one-of-a-kind culture. This is why we say #WhyWorkAnywhereElse?

MasterControl could be your next (and last) career move!

Here are some of the benefits MasterControl employees enjoy:

  • Competitive compensation
  • 100% medical premium coverage (yes, you read that right!)
  • 401(k) plan with company match
  • Generous PTO packages that increase with tenure
  • Schedule flexibility
  • Fitness clubs (you get paid to have fun and be active!)
  • Company parties and employee recognition programs
  • Wellness programs (free Fitbit, gym membership and athletic shoe reimbursements, etc.)
  • Onsite physician and massage therapist
  • Innovation center and gaming rooms at the office
  • Dental/vision plans
  • Employer paid life insurance policy
  • Much, much more!

*** Mention DataYoshi when applying ***

Offers you may like...

  • akapeople

    Senior Data Analyst (m/f)
    Lisboa
  • Dynatrace

    Senior Data Analyst (m/f/x)
    Wien, W
  • McAfee

    Senior Data Analyst
    Madrid, Madrid provincia
  • Harnham

    Senior Data Analyst
    Paris (75)
  • HARNHAM

    SENIOR DATA ANALYST F/H
    Paris 1er (75)