Job Summary: The Associate Data Engineer in Populytics contributes to the development and optimization of `big data' data pipelines, architectures, and data sets for Populytics, as required for population health management, provider profiling, clinical initiatives, medical expense budget tracking, and other applications. Helps maintain complex technology infrastructure and collaborates with other data engineers, clinical & business analysts, and web developers to implement new features and plan for future projects. Associate Data Engineers must learn and use proven design principles, design patterns, and automated testing while helping to build and maintain the data pipeline. They may participate in group meetings with other departments to clarify processing requirements and designs, and they must become knowledgeable on relevant technologies and new industry trends to help sustain a strong technical direction for Populytics.
The Associate Data Engineer contributes to the development and maintenance of an optimal data pipeline architecture, using SQL and HDP big data' technologies, as required for optimal extraction, transformation, and loading of data from a wide variety of data sources, including but not limited to medical and pharmacy claims, HR, lab, EMR, Provider, and Payer systems. Responsibilities include using and supporting appropriate processes and tools for secure, efficient, and reliable data exchange, for data gap analysis and transformation, for data profiling and auditing, for data integration across time periods or data sources, for generating input files and consuming output files from advanced analytics tools, and for feeding the outputs to reporting data marts in a manner that satisfies resource and performance constraints.
Also responsible for contributing to the development and maintenance of audits and monitoring tools that utilize the data pipeline to provide actionable insights into data accuracy, operational efficiency, volume or cost fluctuations, and other key business performance metrics. The Associate Data Engineer contributes to the development of internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, extensibility, and maintainability. Works collaboratively with business partners, including lead and senior data engineers, clinical and business analysts, web developers, and data scientists to assist with data-related technical issues and support their data infrastructure needs. Routinely assists with root cause analyses on internal and external data and pipeline processes to answer specific business questions and identify opportunities for improvement.
Minimum Requirements: Work requires the level of knowledge normally attained through completion of a Bachelor's degree in Software Engineering, Computer Science, or Computer & Information Science.
Minimum Experience: Must have completed a Bachelor?s degree in Software Engineering, Computer Science, or Computer & Information Science and possess working knowledge of programming with object-oriented/object function scripting languages such as Java, Python, C#, and Scala.
Must possess the initiative to identify and carry-out responsibilities to their completion. Strong problem solving abilities and analytical skills. Requires a high degree of professional judgement and inter-personal skills at all levels. Requires the individual to work independently, so as to require minimal supervision. Requires the ability to function under pressure and deadlines.
Preferred Qualifications: Preferred Experience: Strong preference for an individual with working knowledge of SQL and query authoring. Prefer someone who has some familiarity with manipulating, processing and extracting value from large disconnected datasets. Familiarity with relational SQL and NoSQL databases, such as MySQL, SQL Server, Postgres, and HBase is also preferred. Familiarity with data pipeline and workflow management tools such as Oozie, Azkaban, SSIS, and Pentaho, and familiarity with big data tools such as Hadoop, Spark, Hive, Hive LLAP, and Map Reduce Programming are big pluses.
Licensure and Certifications: Hortonworks Certified Associate