eVerge Group

Data Engineer with GCP OR AWS

Job description

Healthcare Data Engineer with GCP OR AWS

6+ month remote contract (C2C or 1099)

Job Description/Job Summary: Responsible for construction and development of large-scale cloud data processing systems The Data Engineer must have considerable expertise in data warehousing and job requires proven coding expertise with Python, Java, SQL, and Spark languages. Must be able to implement enterprise cloud data architecture designs, and will work closely with the rest of the scrum team and internal business partners to identify, evaluate, design, and implement large scale data solutions, structured and unstructured, public and proprietary data. The Data Engineer will work iteratively on the cloud platform to design, develop and implement scalable, high performance solutions that offer measurable business value to customers.

Qualifications and Education: Master level technology degree preferred Technology certifications preferred Bachelor's in computer engineering or equivalent field or equivalent foreign degree required

Required Work Experience:

  • Four to seven years of experience required.
  • Some of the minimum experience requirement may be met with Masters or other advanced degree
  • Cloud Experience Required
  • Coding experience with Python, Java, Spark, and SQL
  • Strong Linux/Unix background and hands on knowledge.
  • Past experience with big data technologies including HDFS, Spark, Impala, Hive,
  • Experience with Shell scripting and bash.
  • Experience with version control platform github
  • Experience unit testing code.
  • Experience with development ecosystem including Jenkins, Artifactory, CI/CD, and Terraform.
  • Works on problems of diverse scope and complexity ranging from moderate to substantial
  • Assists senior professionals in determining methods and procedures for new tasks
  • Leads basic or moderately complex projects/activities on semi-regular basis
  • Must possess excellent written and verbal communication skills
  • Ability to understand and analyze complex data sets
  • Exercises independent judgment on basic or moderately complex issues regarding job and related tasks
  • Makes recommendations to management on new processes, tools and techniques, or development of new products and services
  • Makes decisions regarding daily priorities for a work group; provides guidance to and/or assists staff on non-routine or escalated issues
  • Decisions have a moderate impact on operations within a department
  • Works under minimal supervision, uses independent judgment requiring analysis of variable factors
  • Requires little instruction on day-to-day work and general direction on more complex tasks and projects
  • Collaborates with senior professionals in the development of methods, techniques and analytical approach
  • Ability to advise management on approaches to optimize for data platform success.
  • Able to effectively communicate highly technical information to numerous audiences, including management, the user community, and less-experienced staff.
  • Consistently communicate on status of project deliverables
  • Consistently provide work effort estimates to management to assist in setting priorities
  • Deliver timely work in accordance with estimates
  • Solve problems as they arise and communicate potential roadblocks to manage expectations
  • Adhere strictly to all security policies Desired Work Experience:
  • Proficient in multiple programming languages, frameworks, domains, and tools.
  • Coding skills in Scala
  • Experience with gcp platform development tools Pub/sub, cloud storage, big table, big query, data flow, data proc, and composer desired.
  • Strong Linux/Unix background and hands on knowledge.
  • Knowledge in Hadoop and cloud platforms and surrounding ecosystems.
  • Experience with web services and APIs as in RESTful and SOAP.
  • Ability to document designs and concepts
  • API Orchestration and Choreography for consumer apps
  • Well rounded technical expertise in Apache packages and Hybrid cloud architectures
  • Pipeline creation and automation for Data Acquisition
  • Metadata extraction pipeline design and creation between raw and finally transformed datasets
  • Quality control metrics data collection on data acquisition pipelines
  • Able to collaborate with scrum team including scrum master, product owner, data analysts, Quality Assurance, business owners, and data architecture to produce the best possible end products
  • Experience contributing to and leveraging jira and confluence.
  • Strong experience working with real time streaming applications and batch style large scale distributed computing applications using tools like Spark, Kafka, Flume, pubsub, and airflow.
  • Ability to work with different file formats like Avro, Parquet, and JSON.
  • Managing and scheduling batch jobs.
  • Hands on experience in Analysis, Design, Coding and Testing phases of Software Development Life Cycle (SDLC).

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
August 10, 2022
August 10, 2022
August 10, 2022