- Contribute to business management by effectively solving business-related problems through data modeling/analysis/prediction.
- Establish/support infrastructure that continuously grows to strategically analyze big data.
SRBR is looking for an experienced Data Engineer to help our team to strive for efficiency while implementing and optimizing data pipeline architectures applied to multidisciplinary research. The ideal candidate has experience with pipeline building in scenarios of high volume and high heterogeneity, and is a data wrangler who enjoys optimizing existing data-centered systems or even building them from scratch.
The Data Engineer will help creating and implementing architectures responsible for extracting/ingesting raw data and producing machine-readable datasets that will feed a myriad of models.
Role and Responsibilities
- Propose, implement, and maintain optimal data pipeline architectures;
- Design, implement, install, test, and maintain data management and processing systems;
- Collaborate and offer support to software developers, machine learning engineers, and data scientists on troubleshooting and scaling up data-related initiatives, ensuring optimal, consistent solutions;
- Integrate large, heterogeneous, complex datasets in order to meet functional / non-functional domain requirements;
- Efficiently benchmark, select, and combine technological tools, processing systems, databases, algorithms, and programming languages to fit requirements;
- Plan, deploy and maintain data infrastructures;
- Propose and automate data-quality evaluation processes;
- Plan, install/configure, and implement disaster recovery procedures/tools.
- Solid Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or similar formal education;
- Strong knowledge on developing and deploying data-centered/ETL solutions using Java, Python, NodeJS, Scala, or C/C++;
- Strong knowledge and experience with both software and system architecture design patterns;
- Knowledge and experience with distributed data storage (Hadoop FS) and big data processing engines (Apache Hadoop/MapReduce, Apache Spark, Apache Tez);
- Experience with data ingestion/stream tools (Apache Kafka, RabbitMQ), message protocols (AMQP, MQTT, WebSocket) and ETL tools
- Strong software engineering skills, including methodologies, best practices and related tools;
- Experience applying Agile/Scrum practices and values;
- Experience with data modeling and strong knowledge of SQL/NoSQL DBMSs and query language;
- Proven 3+ years of experience applying required skills in advanced and meaningful set of projects;
- Written and verbal fluency in both Portuguese and English.
Skills And Qualifications
Will be considered a plus:
- Experience working on meaningful artificial intelligence projects;
- Experience with data analysis of high volumes of data;
- Experience working with real-time scenarios like IoT, telemetric, etc.
- Experience working on scenarios involving temporal-series data analysis, statistics, and modeling;
- Bs.C. in Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or other similar titles obtained from top-tier universities.
- Ms.C. or Ph.D. in Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or similar area obtained from a top-tier university (nice to have).
Oportunidade em Campinas/SP
As informações confidenciais e proprietárias obtidas por você durante seu histórico de emprego em outras empresas devem ser mantidas como tal (Confidencial & Proprietária) durante todo o processo de seleção, não devendo, em nenhuma hipótese, ser abertas ou compartilhadas com os entrevistadores.