Job description

On behalf of our client, Procom Consultants Group is currently looking for an Intermediate Data Engineer.


Data Engineer Job Details


  • Knowledge transfer plan development and execution.
  • Development of a statement and plan of services developed for the client pursuant to this


Supplement and that outlines the Support Services to be provided by Supplier and/or Assigned Personnel, the timelines for the delivery of the Support Services, approval process and acceptance criteria for the Support Services, etc. (a “Statement of Work”).
  • Execution of tasks under Statement of Work, once accepted by the client, in accordance with the set processes and procedures under the Statement of Work.
  • Analyze the data and develop Scala/Spark programs and related components in the work areas of Information Management (IM) relating to claims.
  • Use data pipelines to extract data from data sources which are in various formats (viz - flat files, XML, relational tables, Oracle logs), and use tools (such as StreamSets, Scala and Spark programs) to transform and store the data in big data platform (Data Lake) after data validation.


Deliverables


Assigned Personnel shall provide the following “Deliverables” with the stated delivery dates which may include documentation, reports, software code, programming code, manuals, designs, drawings, schematics, and diagrams:


  • Develop Stream Sets, Spark and Scala programs for data ingestion.
  • Data transformation as per the mapping document.
  • Data Analysis as per requirements and develop data models/mapping.
  • DataValidation.


Data Engineer Mandatory Skills


Communication & Problem Solving Skills:


  • Communicating & presenting solutions to internal developers
  • Ability to apply critical thinking skills to develop efficient code, data transformation logic and solution design
  • Experience and/or understanding of Agile Methodologies


Technical Skills:


  • Advanced Scala programming skills
  • Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)
  • Extensive programming experience in either Java, C++ or preferably Scala, IntelliJ, Jenkins
  • Working experience with Apache Spark Data Processing Framework, Parquet data format
  • Good understanding of Relational (Oracle/DB2) and Hadoop Big Data platforms
  • Experience working with Jira


Data Engineer Start Date


ASAP


Intermediate Data Engineer Assignment Duration


March 31 2023


Intermediate Data Engineer Working Location


Fully remote but PST hours

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
MedPro Group
November 26, 2022
Kamo Placements
November 26, 2022

Product Data Engineer - 7 Years Contract

Phanda Personnel
November 26, 2022

Product Data Engineer