Data Engineer

Company:
Location: Remote

*** Mention DataYoshi when applying ***

Hagerty, an automotive lifestyle brand for people who love cars and love driving them, has an opportunity for a Data Engineer to join on our Enterprise Data Hub team.

As a Data Engineer, you will be joining a fast-paced, high functioning team to build and maintain data pipelines and services that support Hagerty’s Enterprise Data Hub (EDH). Internally engineered and developed, the EDH includes data processing & storage, services, APIs, and Hagerty’s internally facing “Data Portal” web app. In this role you will develop data pipelines, services, and cloud-based infrastructure to support the growth of Hagerty’s Insurance business and automotive lifestyle brand.

You will be partnering with a team of talented engineers working in an agile environment, leveraging modern cloud-based technologies to drive data-driven decision making in analytics and Hagerty’s data products. Remote work is an option for this role.

What You'll Do:

  • Implement best practices around software development and big data engineering
  • Develop and implement robust and scalable data pipelines using Python, SQL, parallel processing frameworks, and other AWS/Salesforce cloud solutions
  • Develop and implement batch data pipelines using tools such as Apache Airflow, Snowflake, and numerous AWS products (EC2, Fargate, ECS, Lambda, and RDS)
  • Develop streaming data integrations to support products across the Hagerty portfolio and support real-time reporting
  • Develop Enterprise Data Hub platform infrastructure using Terraform infrastructure-as-code
  • Develop and support Hagerty’s cloud-based data warehouse to enable analytics and product reporting
  • Partner with internal and external stakeholders to collect requirements, recommend best practice solutions, and productionize new data ingestions/analytic workloads
  • Develop solutions to catalog and manage metadata to support data governance and data democratization
  • Partner with Data Quality Engineers to define and implement automated test cases and data reconciliation to validate ETL processes and data quality & integrity
  • Mentor junior team-members in software and big data engineering best practices
  • Partner with Data Scientists to design, code, train, test, deploy and iterate machine learning algorithms and systems at scale

This Might Describe You:

  • Associates degree, preferably in a technical/analytical field, or relevant work experience
  • Additional 3+ years working in another role within an IT delivery team, such as a developer, engineer, data analyst, quality assurance analyst, ETL developer or DBA
  • Strong problem-solving abilities and attention to detail
  • Ability to authentically and effectively communicate (written and verbally) with various stakeholders
  • Ability to create technical artifacts and documentation to support development and maintenance of data products
  • Experience in successful delivery of data products as productionizable software solutions
  • Proven development experience of back-end or data solutions using Python
  • Experience or willingness to learn open-source data processing technologies such as streaming services (Kafka / SQS), big data processing frameworks (MapReduce/Spark), big data file stores (EMRFS / HDFS)
  • Experience ensuring rigorous code development, testing, automation, and other software engineering best practices
  • Experience in imperative (e.g., Apache Airflow / NiFi) or declarative (e.g., Informatica/Talend/Pentaho) ETL design, implementation, and maintenance.
  • Experience cataloging and processing non-relational data
  • Functional knowledge of relational databases and query authoring (SQL).
  • Experience with developing infrastructure as code in a cloud-based environment (Terraform experience preferred)
  • Experience with container-based development preferred
  • Experience working with evaluating different data containers based on workload needs (JSON, delimited files, Avro, Parquet)


If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

*** Mention DataYoshi when applying ***

Offers you may like...

  • RBC

    Data Analyst/Data Engineer, Internal Audit, Fall 2...
    Toronto, ON
  • USAA

    Data Engineer Lead (Bank) Remote
    San Antonio, TX 78288
  • Cognizant Technology Solutions

    Data Engineer
    Tampa, FL 33601
  • Interior Define

    Data Engineer
    Remote
  • Ursus

    Data Engineer- ETL and Architecture
    Menlo Park, CA