Data Engineer

On behalf of our client, Procom Consultants Group is currently looking for an Intermediate Data Engineer.

Data Engineer Job Details

• Knowledge transfer plan development and execution.
• Development of a statement and plan of services developed for the client pursuant to this

Supplement and that outlines the Support Services to be provided by Supplier and/or Assigned Personnel, the timelines for the delivery of the Support Services, approval process and acceptance criteria for the Support Services, etc. (a “Statement of Work”).

• Execution of tasks under Statement of Work, once accepted by the client, in accordance with the set processes and procedures under the Statement of Work.
• Analyze the data and develop Scala/Spark programs and related components in the work areas of Information Management (IM) relating to claims.
• Use data pipelines to extract data from data sources which are in various formats (viz - flat files, XML, relational tables, Oracle logs), and use tools (such as StreamSets, Scala and Spark programs) to transform and store the data in big data platform (Data Lake) after data validation.

Deliverables

Assigned Personnel shall provide the following “Deliverables” with the stated delivery dates which may include documentation, reports, software code, programming code, manuals, designs, drawings, schematics, and diagrams:

• Develop Stream Sets, Spark and Scala programs for data ingestion.
• Data transformation as per the mapping document.
• Data Analysis as per requirements and develop data models/mapping.
• DataValidation.

Data Engineer Mandatory Skills

Communication & Problem Solving Skills:

• Communicating & presenting solutions to internal developers
• Ability to apply critical thinking skills to develop efficient code, data transformation logic and solution design
• Experience and/or understanding of Agile Methodologies

Technical Skills:

• Advanced Scala programming skills
• Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)
• Extensive programming experience in either Java, C++ or preferably Scala, IntelliJ, Jenkins
• Working experience with Apache Spark Data Processing Framework, Parquet data format
• Good understanding of Relational (Oracle/DB2) and Hadoop Big Data platforms
• Experience working with Jira

Data Engineer Start Date

ASAP

Intermediate Data Engineer Assignment Duration

March 31 2023

Intermediate Data Engineer Working Location

Fully remote but PST hours