Data Engineer Lead

Job Location

Education Level

Travel Percentage

Job Category

POSITION SUMMARY:

The Data Engineer Lead role will join HealthStreams collaborative work environment as a leader in the design, implementation, and maintenance of a large-scale data intelligence solution. This person is passionate about big data, being technically proficient and has a deep sense of ownership.

The Data Engineer Lead should be an expert in modern data platform architectures, comfortable building optimized data systems from the ground up, strong familiarity with data science and machine learning principles, pipelines, MLOPs. The Data Engineer Lead will bring innovative thought into cross-team, cross-functional conversations for product strategies, determining priorities and platform solutions.

In this management position, the Team Leader will manage a team of data engineers. Team Leaders have leadership responsibility to evaluate and make recommendations on candidates for data engineering positions. Team Leaders manage department meetings. Team Leaders have a responsibility to help enforce and uphold coding and data engineering standards and guidelines for the team in accordance with HealthStream practices.

ESSENTIAL DUTIES OR RESPONSIBILITIES

• Actively mentor and support career growth of individuals on your team
• Ability to manage senior relationships across all the Business and Functional areas
• Provide technical leadership towards architecting and delivering end to end solutions
• Collaborate on creating and maintaining a flexible, secure, compliant, and optimal data platform architecture
• Maintain/Optimize data pipelines using various data/big data/ETL technologies (batch and real-time) system will have metrics/KPIs to determine system health and performance
• Construct tools for data analytics & data science undertakings - creating an industry leading product
• Work with Business and Product stakeholders to refine and inform the requirements for data platform features and enhancements providing complexity estimates
• Develop/incorporate/explore complex datasets that meet functional and non-functional requirements
• Author and maintain automated processes and tests to ensure system/data integrity and accuracy
• Build flexible integrations between applications and visualization systems
• Develop architecture and design patterns to process and store high volume data sets
• Identify and resolve issues concerning data management to improve data quality
• Implement automated workflows and routines using workflow scheduling tools
• Build continuous integration, test-driven development, and production deployment frameworks
• Adhere to all HealthStream standards, processes, and best practices for coding and data engineering

JOB SPECIFICATIONS:

1.. KNOWLEDGE/EDUCATION AND EXPERIENCE REQUIRED:

• Bachelors degree in IT or related field. At least three years of experience with software development/data solutions in the Cloud.
• 5+ years of relevant experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
• 4+ years' experience in Big Data stack environments and processing tools (Hadoop, SPARK, Hive, Kafka, Kinesis, Delta Lake or similar)
• Strong experience with relational and non-relational SQL (Cassandra, MongoDB, DynamoDB or similar)
• Experience in building real-time streaming data pipelines
• Experience working on CI/CD processes and source control tools such as GitHub and related dev processes
• Willingness to pick up new programming languages, technologies, and frameworks
• Experience with BI/analytics/visualization tools (Metabase, QueryTree, Looker, Power BI or similar)
• Beneficial experiences/knowledge:
  • Experience with Infrastructure automation (Terraform or similar)
  • Experience with container and caching technologies
  • Experience with Glue, SSIS or similar
  • Data Governance
  • HIPAA compliance
  • GDPR

2. SKILLS REQUIRED:

• Strong analytic skills working with structured and unstructured datasets
• Strong experience in coding languages (e.g., Java, Scala, Python)
• Cloud Services and Concepts (AWS, Azure or similar)
• Advanced working knowledge of Relational Database technologies (MySQL, Postgres ..), data models, data mining, segmentation techniques and optimization
• ReST API design
• Effective communication skills (with technical team members, the business, documentation, and in code)

3. ABILITIES REQUIRED:

• Accountability - Taking responsibility for ones actions and the consequences that result from those actions
• Communication & Positive Presence - Listening attentively and respectfully; expressing thoughts and ideas effectively and conciselyboth verbally and in writing. Exhibiting a pleasant attitude
• Collaboration - Cooperating and working effectively with others in the pursuit of common goals
• Critical Thinking - Using sound judgment, logic, and common sense to identify efficient and effective ways of approaching problems or new initiatives
• Adaptability - Being flexible and continuing to function effectively during times of change
• Continuous Improvement - Keeping an open mind; seeking new ideas and ways of doing things to streamline processes and increase client/customer satisfaction
• Planning & Organizing - Appropriately prioritizing activities, scheduling time, and utilizing resources to complete tasks in an efficient and timely manner while paying attention to details
• Work Ethic - Setting high standards for ones work; taking care to ensure all aspects of work are completed accurately
• Customer Service - Treating clients/customers with a high level of respect and taking action in order to meet or exceed expectations and resolve problems
• Motivation - Highly motivated individual with a genuine interest in improving and creating solutions