Data Engineer
Data Engineer
Company: GoPuff
GoPuff
Location: Philadelphia, PA
*** Mention DataYoshi when applying ***
Gopuff is seeking a Data Engineer which will be responsible for supporting the data ingestion and integration needs at Gopuff. The data engineer must provide technical expertise and leadership on architecture, design, and integration of multiple datasets into Gopuff’s ever evolving, large scale data environment. Emphasis of this position will be in developing and deploying a robust data processing pipeline and streams.
Qualifications
- 2-4 years of experience in a Data Engineering role
- Experience building, maintaining, and improving Data Processing Pipeline / Data routing in large scale environments
- Fluency in common query languages, API development, data transformation, and integration of data streams
- Strong experience with large dataset platforms such as (e.g. Amazon EMR, Amazon Redshift, Elasticsearch, Amazon Athena, Azure SQL Database, Azure Database for PostgreSQL, Azure Cosmos DB)
- Experience with data flow monitoring, batching, and streaming
- Experience working with variety of databases and expert using SQL
- Experience data storage performance analysis and enhancements such as data replication, distribution, and compression
- Creativity to go beyond current tools to deliver the best solution to the problem
- Data QA experience is a plus
- Experience with event-based and transaction-based system is a plus
- Experience in producing and consuming topics to/from Apache Kafka, AWS Kinesis, or Azure Event Hubs is a plus
- Experience with acquiring data from varied sources such as: API, data queues, flat-file, remote databases is a plus
Responsibilities
- Responsible for designing, deploying, and maintaining analytics about Gopuff’s environment that process data at scale
- Contributes design, configuration, deployment, and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation, enrichment, and loading of data into a variety of cloud data platforms, including AWS and Microsoft Azure
- Identifies gaps and improves the existing platform to improve quality, robustness, maintainability, and speed
- Evaluates new and upcoming big data solutions and makes recommendations for adoption to extend our platform to meet advanced analytics use cases, such as predictive modeling and recommendation engines
- Performs development, QA, and DevOps roles as needed to ensure total end to end responsibility of solutions
The only predictable thing about life is that it’s wildly unpredictable. That’s where we come in.
When life does what it does best, customers turn to Gopuff to deliver their everyday essentials, and to get through their day & night, work day and weekend.
We’re assembling a team of thinkers, dreamers & risk-takers...the kind of people who know the value of peace of mind in an unpredictable world. (And people who love snacks.)
Like what you’re hearing? Welcome to Gopuff.
The Gopuff Fam is committed to an inclusive workplace where we do not discriminate on the basis of race, sex, gender, national origin, religion, sexual orientation, gender identity, marital or familial status, age, ancestry, disability, genetic information, or any other characteristic protected by applicable laws. We believe in diversity and encourage any qualified individual to apply. We are an equal employment opportunity employer.
*** Mention DataYoshi when applying ***
Offers you may like...
Signpost
Data Engineer
Colorado
Fivesky
Python Developer, Data Engineer
New York, NY
Business Transformation Institute
P1-042621-3 Data Engineer (SE6)
Columbia, MD 21044
Businessolver
Data Engineer
Illinois
Piper Companies
Data Engineer (SQL/Snowflake)
Remote