Finning

Data Engineer III

Job description

Company:

Finning International Inc.

Number of Openings:

2

Worker Type:

Permanent

Position Overview:

Data is deeply embedded in the product and engineering culture at Finning, to maximize productivity and safety of Finning engineers and our customers. The Senior Data Engineer will work with data engineers on building a state-of-the-art data foundation to solve real world business challenges and optimizing Finning processes across a wide range of areas including supply chain, marketing, pricing, and sales. Working alongside a first-class Data Science team, this position will assist the data architect and data engineering lead in designing and exploring reliable and scalable data-driven solutions for optimizing maintenance, inventory, fuel, scheduling, and other services for Finning customers. The senior data engineer will rely on extensive background and experience in data modeling and building traditional and distributed data pipelines to help guide and explore new grounds in adopting new tools to improve Finning’s data processing process and infrastructure. The Senior Data Engineer will work with the Analytics management team to provide guidance to Data Engineers and set work priorities collaboratively with other Finning teams as we continue to explore new areas inside and outside of business. This position will work with machine health and productivity data received from over 25,000 connected equipment, transactional data for Finning sales and services, customer relationship data, marketing data, and public data sources such as weather and macroeconomic factors.

Job Description:

Major Job Functions:
  • Develop and maintain data integration processes that can consume data from many different sources in many different formats. (20%)
  • Design and develop reusable data pipeline components like Azure functions and logic apps.
  • Research and resolve unstructured data ingestion mechanisms.
  • Be invested in building data pipelines that ensure data quality by incorporating QA processes and tools to test and monitor data quality and raise alerts for data quality problems. (20%)
  • Write test plans and develop data pipeline unit tests
  • Design and contribute to the QA automation efforts so testing can be part of CI/CD process.
  • Create task specifications from business requirements and break them down into sub-tasks so work can be distributed among data engineers. (10%)
  • Provide high level but reasonable estimates for tasks and sub-tasks.
  • Prioritize and guide data engineers to work as a team to optimize development effort.
  • Investigate data quality problems and propose viable solutions in fixing these problems. (10%)
  • Instrument monitoring tools to measure performance and detect processing issues
  • Be familiar with the environment data pipelines are in (e.g. network configuration) so problem diagnosis can be more comprehensive.
  • Conduct design and code reviews with data engineers. (10%)
  • Coach data engineer to prefer reusable code and adhere to development standards.
  • Work with the business and non-technical stakeholders to discover and document technical requirements and understanding the core problem. (10%)
  • Be emphatic to business stakeholders’ non-technical issues and willing to adapt technical design and requirements to suite business needs.
  • Provide timely updates both to business stakeholders and to the DAS team when requirements change.
  • Drive the continually improvement of ongoing reporting and analysis processes, automating and/or simplifying self-service support. (10%)
  • Help foster the culture of recording issues and actively follow up and resolve long term problems and / or systematic issues.
  • Stay abreast of innovations in Cloud infrastructure, Business Intelligence, Analytics and Data Warehouse tools and technologies. (10%)
  • Help coach and mentor data engineers to be lifetime learners.
  • Help identify and organize training opportunities like lunch and learn and various on-line webinars.

Accountabilities:
  • Delegate work to data engineers but responsible for overall delivery of features while ensuring data quality and pipeline stability.
  • Design, develop, and maintain data pipelines
  • Gather requirements, design and maintain the delivery roadmap, and deliver results
  • Create automated metrics using complex distributed databases and sources
  • Be able to delegate and support a small team to work on large technical tasks from inception to completion.
  • Present technical plan to and implement feedback from cross functional teams across a Finning Digital and IT organization

Education & Experience:
  • BS degree in Computer Science, Engineering, or equivalent. MS degree preferred.
  • Minimum of 5 years’ experience coding in two of C#, Java or Python
  • Minimum of 5 years’ experience working with relational databases and the SQL language
  • Minimum of 5 years’ experience developing data injection, consumption, ETL, data sanitization and developing and maintaining the whole cycle of a data warehouse.
  • Ability to create SSIS packages that can handle ETL of large amount of data coming from different sources is preferred.
  • Minimum of 2 years in a technical lead/mentoring position coordinating a technical team.
  • Experience consuming, transforming, cleaning and persisting large amounts of data.
  • Extensive experience with relational database technology and the SQL language. Preference is given to SQL Server and T-SQL.
  • Experience working with data warehouse and maintaining the whole cycles from gathering requirements to delivering facts and dimensions to stakeholders
  • Experience consuming and creating RESTFUL API’s
  • Experience in infrastructure automation/orchestration is a plus
  • Experience with public cloud platforms (AWS, Azure, GCP) and a solid understanding of the tradeoffs between PaaS and IaaS
  • Experience with Hadoop, Hive, Kafka and other big data tools and distributed systems is a plus
  • Able to write SQL scripts for analysis and reporting (SQL, MySQL)
  • Familiar with configuration management systems like Puppet, Chef, Salt, Ansible

Knowledge:
  • Data warehousing
  • Relational and dimensional modelling
  • Data replication and ingestion
  • Data pipeline design, QA best practices and cloud infrastructure
  • Containers and container application platforms like K8s and Docker
  • Networks, storage, systems architecture
  • CI/CD build systems
  • Monitoring systems & Log Collection / Aggregation

We are committed to diversity at Finning, to building and sustaining a diverse and inclusive workforce and as an equal opportunity employer we encourage applications from all qualified individuals. Finning does not discriminate against applicants based on genders, races, national and ethnic origins, religions, ages, sexual orientation, marital and family status, and/or mental or physical disabilities.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.