Job description

About Us

Healthdirect Australia is a government-owned, not-for-profit organisation who works with purpose and dedication to help Australians actively manage and improve their health.

Our virtual services are easily accessible via multiple channels, including helplines, video call solutions, apps and websites and cater for Australians at all stages of life, ranging from practical and emotional support during pregnancy through to advice about aged care and helping Australians through the COVID-19 pandemic.

About The Opportunity

The Senior Data Engineer will be a part of the Technology division supporting our National Health Services Directory (NHSD). The NHSD provides information on a range of health services, including general practices, allied health, hospital, and community services to support health professionals to deliver care and consumers to access health service information, including to make an appointment at a health service.

In this role, the Senior Data Engineer creates updates, tests, and reviews data solutions that result in meaningful improvements to the Directory that delivers value to Healthdirect stakeholders in line with our Corporate Plan. The NHSD utilises a “large-scale data processing system “and this role is involved in the design, build and delivery of that system.

Key Responsibilities Include
  • Develop data re-engineering processes, including identity allocation, resolution, cleansing and reconciliation.
  • Design, develop, test, deploy and automate ETL/ELT solutions
  • Gather and process raw data at scale (including writing scripts, invoking APIs, developing SQL/SQL dialect queries, etc.).
  • Process structured/unstructured data into a form suitable for analysis - and then do the analysis.
  • Support reporting requirements by analysing, extracting, and publishing information on anomalies, insights, trends.
  • Design, build, and maintain our data warehouse/data lake of matching data and health sector raw data files
  • Design, build and maintain code that transforms the data into useful formats
  • Scale up solutions in distributed computing frameworks such as MapReduce and Spark.
  • Work closely with Data Management Team, Software Engineers, and Architects to develop, maintain and enforce data management and security policies.
  • Work closely with our other development teams to integrate your designs, innovations, and algorithms into production systems.
  • Work with business stakeholders and development teams to prioritize stories, resolve conflicts, develop design criteria, and building requirements for successful delivery
  • Design service-level and data-level metrics and monitoring to indicate quality, detect problems and identify inappropriate usage; and create mechanisms to visualize these
  • Assist with testing to ensure services perform and meet non-functional requirements
  • Act as a subject matter expert for the product teams on data processing solutions and data management.
What we’re looking for
  • 4+ years’ recent experience in Scala and production level exposure to at least one other programming or scripting language such as Java, Python and/or R.
  • 3+ years of strong experience with Apache Spark and Spark Streaming is mandatory.
  • 5+ years in ETL/ELT implementations.
  • 3+ years’ experience with database design and data modelling.
Why work with us

We believe our strength comes from our talented and dedicated staff. We work hard to deliver an environment where you can enjoy your role and are rewarded for it, from offering competitive remuneration packages through to providing the opportunity to make a daily difference in the lives of others. Some of our other benefits include flexible work arrangements (2 days per week in office requirement for full time employees and pro-rated for part time employees) to help you balance work and life; 14 weeks company Paid Parental Leave for all parents and other parental leave entitlements to support families; and access to formal development opportunities and workplace learning that can expand your capabilities and unlock your talent.

To find out more about Healthdirect, and our vision for a healthier Australia, please visit our website -

As an Equal Opportunity Employer, we respect and seek to empower each individual and support the diverse cultures, perspectives, skills, and experiences within our workforce.

Click Apply to submit your application for this role. Please be aware that you must have the right to live and work in Australia to apply for this job

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
January 31, 2023

Senior Data Engineer

Wood Mackenzie
January 31, 2023

Senior Data Engineer

Commonwealth Bank
January 31, 2023

Senior Data Engineer