Position: Sr. Data Engineer
Reports to: Director of Engineering
DMG is seeking a Sr. Data Engineer to implement efficient data pipelines supporting business intelligence, analytics, and operational needs. This role will be proficient in investigating, transforming, and combining data of different data sources and types. Ideal candidates are prepared to design appropriate architecture to support business data needs and create consumable analysis-ready data sets. The Sr. Data Engineer will educate business partners on the use and sources of data. They will identify and explore new data sources. Assignments may include supporting Data Science, Data Governance, integrations and/or deployments with Product & Engineering and other technical professionals, overseeing support of other engineers, and collaboration with business partners within and across business teams.
- Creates and maintains optimal data pipeline architecture for extraction, transformation, and loading of data from various data sources – both internal and external
- Builds analytical tools to utilize the data pipeline
- Assembles large, complex sets of data that meet non-functional and functional business requirements
- Builds industrialized analytic datasets and delivery mechanisms that utilize the data pipeline to deliver actionable insights
- Identifies, designs, and implements internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Contributes to the architecture and design for scalable and efficient model deployments
- Investigates data, data sources and performs data quality analysis
- Communicates and maintains master data, metadata, data management repositories, logical data models, data standards
- Works with stakeholders including the executive, product, data and design teams to support their data infrastructure needs while assisting with data-related technical issues
- Establishes and teaches best practices related to data access and queries for data users
- With minimal direction, plans, implements, manages, and/or contributes on projects that are moderate to high complexity and scale using accepted project management standards
- Trains/coaches team members or peers and actively shares expertise with peers
- 5+ years of hands-on experience with Big Data Tools: Hadoop, Spark, Kafka, etc.
- Mastery of relational SQL and NoSQL databases, including Postgres and Cassandra
- Experienced with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experienced with Stream-processing systems: Storm, Spark-Streaming, etc.
- 5+ years with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Proficiency with geospatial data: latitude/longitude, geopoint/vector containers, geohash, H3, shapefiles, etc.
- Experienced with data wrangling and preparation for use within data science, actuarial, business intelligence or similar analytical workflows
- A demonstrable understanding of networking/distributed computing environment concepts
- Strong written and verbal communication skills including the ability to effectively collaborate with multi-disciplinary groups and all organizational levels
- High level organizational and project management skills in order to handle multiple concurrent assignments in a timely manner and to monitor processes throughout a team or department
- Strong decision-making skills
Education: Master’s degree or equivalent experience in information technology, computer science, data science, engineering, mathematics, statistics/applied statistics, or related field
DMG is an equal opportunity employer.