Data Engineers (PySpark) - Large Fin. Services - 6...

• 6/12 Month Contract
• Sydney CBD

Consortio Sourcing is searching for an experienced Data Engineers with PySpark experience to work on a new initiative for a Major Financial Services organization based in Melbourne. This is an exciting opportunity for experienced to work with a talented and bright team who are committed to delivering world-class technology solutions for their customers.

Key Responsibilities

• Design, construct, install, test and maintain data pipelines across various layers in the data lake and presentation layer
• Employ pyspark, sql in the code. Experience with a data processing engine like Databricks or EMR or equivalent
• Collaborate with members of your team (eg, data architects, the IT team, data scientists) on the project's goals.
• Recommend different ways to constantly improve data reliability and quality.
• Devops knowledge and experience - pipelines and cicd
• Generic pipelines and frameworks for jobs and ingestion, curation etc

Overview

• Experience of building data and analytics platforms with structured and un-structured data sources
• Design, construct, install, test and maintain data pipelines across various layers in the data lake and presentation layer
• Build high-performance algorithms, and prototypes around ETL data movement and ML models
• Hands-on experience with unit testing automation, regression testing suite and automated test data generation
• Employ pyspark, sql in notebooks. Experience with a data processing engine like Databricks or EMR or equivalent
• Collaborate with members of team (eg, data architects, the IT team, data scientists) on the project's goals
• Recommend different ways to constantly improve data reliability and quality
• Data modelling experience - Kimball, Inmon, vault models
• Devops, git integration, automation of ci/cd pipelines
• Agile processes
• Implementation of Streaming analytics solutions
• IoT, Event based triggers(Pub-Sub) systems
• Azure experience

On offer is a 6-month contract paying c.$900.00 p.d inc. super and the opportunity to work on a high profile project for a large well known Fin. Services customer in the CBD.