As a Data Engineer you will build curated datasets and make them accessible to our partner teams by writing at scale production data pipelines. Your work will enable the decision-makers across the service operations organization to bring together insights and support our operations strategy and customer experience. In every decision that you influence, you will see the product safety improve and be more valuable to Twitter users.
In this role you will:
- Design, develop, and launch extremely efficient and reliable data pipelines to move data and to provide intuitive analytics to our partner teams.
- Make Twitter-scale data more discoverable and easy to use for Data Scientists and Analysts across the service operations organization
- Collaborate with other engineers and Data Scientists to discover the best solutions.
- Diagnose and solve issues in our existing data pipelines and envision and build their successors.Qualifications
- Good understanding of one or more of the following: Python, Scala, or Java
- Strong understanding of SQL
- Broad knowledge of the data infrastructure ecosystem
- Experience with Hadoop or other MapReduce-based architectures
- Experience working with large data volumes
- Experience in building Data Warehouses and data modeling.
Experience with any of the following is a plus:
- BigQuery, Presto, or Hive
- Scalding, Spark, DataFlow, or Airflow2:32Please let me know if this looks fine.