Senior Data Engineer

Job description

Company Description
Swiftly is a mission-driven company and a big data platform for transportation. The platform weaves data analytics, intuitive visualizations, and real-time predictive technologies into applications that drive smarter decisions for public transit agencies. Today, over 5,500 transit professionals in over 100 cities across the globe use Swiftly to improve transportation for over 1.5 billion passenger trips per year.

Engineering at Swiftly
Engineering at Swiftly is not only about writing code – we believe in creating empowered product teams that work together to conceptualize new features and bring them to life. Each team aims to strike a balance between delivering incremental improvements, creating prototypes to test new ideas and mitigate risks, and building scalable software using industry best practices. We’re guided by a mission to positively impact transit riders, and we embrace humility and intentionality in how we make technical decisions so that we best meet our customers’ needs.

About the Role
As our first dedicated data engineering hire, you will help us make sense of our vast historical transit data by designing systems that make data accessible to individuals and available in our product. You will be a member of our historical data product team that owns our Insights product. You will contribute to product solutions that help planners, schedulers, and other individuals at transit agencies optimize future transit service by using our historical raw data, such as GPS, and derived data, such as on-time performance, headways, runtimes, and speeds.

You'll collaborate with your teammates and with engineers on an adjacent team that owns the analysis and storage of real-time transit data. We are looking for someone who has experience with a variety of data platforms and knowledge of the data ecosystem at large, in order to get up to speed on our existing systems and help identify the right solutions to unlock opportunities with our data.

We encourage you to apply if you meet half or more of the below requirements and are excited to learn about the rest. Above all else, we value candidates who have a positive attitude and the aptitude to learn the requisite skills. We know the tech landscape is ever-changing and strongly support individual growth. If this is exciting to you, then we'd love to talk.

Swiftly's office is located in San Francisco, CA. We are open to local or US based remote hires for this role. We are not able to provide visa sponsorship at this time. #LI-Remote


    • Design the architecture of data pipelines which enable internal self-service exploration of and prototyping with historical transit data
    • Build data systems accessible via API that are designed to respond with low latency to queries within our Insights product line
    • Acquire deep knowledge of our core database schemas and the data within to assist in architectural planning and database optimization & design in the future
    • Apply principles of observability to ensure data systems remain highly available with a low mean-time-to-recovery by incorporating monitoring, logging, and alerting
    • Build and manage a framework for running scheduled or event-driven jobs that teams can extend for a variety of use cases
    • Assist product teams in capturing user analytics and building data pipeline(s) to surface user insights in business intelligence tools

You'd be a good fit if...

    • You have 5+ years of software engineering experience and have a data engineering background
    • You have a data engineering background and a breadth of knowledge of data platforms (e.g. S3, Spark, Druid, etc.) so you can evaluate new technologies and select the right tool for the job, with an eye toward robust, scalable, and flexible architectures
    • You have written ETL pipelines or other infrastructure to sync data between systems
    • You are excited to be part of a mission-driven company. Regardless of your why, it is important that you care about the work we do
    • You love solving problems with data
    • You are a strong communicator who can partner with and help mentor other engineers to become self-sufficient in building and maintaining data pipelines at Swiftly
    • You embrace a mindset of continuous learning – helping both yourself and those around you adapt as technology constantly evolves
    • You are a dynamic team player and seek to make valuable contributions wherever it is most-needed at any given time
    • You’re open to receiving feedback (a core Swiftly value!), approach technical debates with humility, and participate in blameless postmortems when things go wrong.
    • Bonus if familiar with our tech stack: Postgres/RDS, Druid, AWS Athena, AWS Glue (Spark jobs), AWS Step Functions (jobs)
COVID-19 Vaccination Policy

As a tight-knit company where we have the utmost concern and care for our team members, Swiftly has adopted a mandatory COVID vaccination policy.

This policy will comply with all applicable laws and is based on guidance from the Centers for Disease Control and Prevention and local health authorities.

We will not require any candidates to be vaccinated in order to interview for our open positions, but all new hires at Swiftly will be required to share a copy of their vaccination certification or present documentation to request an exemption.

In compliance with the EEOC, any team member in need of an exemption from this policy due to a medical reason, or because of a sincerely held religious belief must provide Swiftly People Ops with appropriate documentation. For a medical exemption, we require a doctor’s note, and for a religious exemption, we require a note from the team member outlining the request.

  • Competitive salary
  • Stock options for every employee
  • Medical, Dental and Vision
  • 401k with Employer Match
  • Flexible Spending Account (FSA)
  • Home office setup reimbursement
  • Monthly cell/internet reimbursement
  • Monthly Education Reimbursement
  • Unlimited PTO
  • Flexible work environment
- 17 paid holidays - including 4 holidays in months without US national holidays in 2021
  • 8 fully paid weeks of leave for child birth/adoption

Don’t just hit the apply button. We want to hear more about you. Tell us why you're passionate about our mission and what interests you about Swiftly!

We are an equal opportunity employer - we are committed to a workplace that is as dynamic, diverse, and passionate as the communities we serve.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.