GeekHunter Brasil

Data Engineer Sênior Spark (Remoto)

Job description

Esta é uma vaga de um cliente da GeekHunter, candidate-se para ter acesso às informações completas sobre a empresa.


Responsibilities


  • Build large-scale batch and real-time data pipelines with data processing frameworks like Spark and AWS managed services.
  • Use best practices in continuous integration and delivery.
  • Help drive optimization, testing and tooling to improve data quality and our ability to use data to make product decisions.
  • Collaborate with other software engineers, ML engineers, Data Scientists, Data Analysts, and stakeholders, taking learning and leadership opportunities that will arise every single day.
  • Collaborate with the Analytics function to support their BI tools and initiatives to deliver the reliability, speed, and scalability of a data platform they’ll love working with.
  • Work in multi-functional agile teams to continuously experiment, iterate and deliver on new product and infrastructure objectives.


Optimize the existing data warehouse which will create a single version of the truth and standardize data into coherent formats for self-service


  • Keep abreast of new data storage, delivery, analysis, visualization, reporting techniques and software to develop more powerful data infrastructure.
  • Be a trusted technical advisor to customers and solve complex data challenges.
  • Inspire and lead others with your work ethic, business results, intrapersonal skills and willingness to see success based on team accomplishments vs. your individual achievements.
  • This role will report to the Data Engineering Manager


Description


  • We're looking for a Senior Data Engineer with know how to work with high volume heterogeneous data, preferably with distributed systems on the AWS. - English is common on a daily basis, so the ability to conduct technical and complex conversations in English is a must.


Technical Skills


  • Know how to work with high volume heterogeneous data, preferably with distributed systems on the AWS platform.
  • Know how to write distributed, high-volume services in Go, Java, Scala, or Python leveraging AWS managed services.
  • Knowledgeable about data modeling, data access, and data storage techniques.
  • Appreciate agile software processes, data-driven development, reliability, and responsible experimentation.
  • Want to own the software you write in production.
  • Understand the value of partnership within and across teams.
  • Care a lot about fostering a diverse culture that includes everyone and supports them being their authentic self. We strongly believe that diversity of experience, perspectives, and background will lead to a better environment for our employees and a better product for our customers.
  • Experience building the infrastructure required for optimal ETL of data from a wide variety of data sources.
  • A successful history of manipulating, processing and extracting value from large disconnected datasets.
  • Superior understanding of database query languages and substantial knowledge in analytical approaches.


Required Qualifiquations


  • Bachelor’s degree in a technology-related field (data science, computer science, software engineering, etc.), with 5+ years’ data engineering experience.
  • Hands-on experience with data processing software and algorithms.
  • Experience in writing software in one or more languages: Java, Python, Go. Experience in SQL.
  • Experience managing client-facing projects, troubleshooting technical issues, working with cross-functional stakeholders.
  • Experience in working with/on data warehouses, including data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures.
  • Excellent verbal and written communication skills.
  • Reflective, independent and eager learner (e.g., learns from mistakes, asks good questions, able to generate creative solutions to problems with minimal guidance)
  • High levels of integrity, autonomy, self-motivation and ability to work well in a team.


Habilidades Obrigatórias


  • Python
  • Spark


Habilidades Desejáveis


  • Kubernetes
  • Snowflake
  • AWS


A vaga aceita trabalho remoto?


Sim

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
The Craneware Group
December 6, 2022
APRIL
December 6, 2022

Data Engineer