Techstra is seeking an experienced Senior Data Engineer / Spark Developer to work closely with the business, analytics and IT teams to support the design, develop, and sourcing of data into a Hadoop environment (Hortonworks).
From the day you join, you will hit the ground running surrounded by amazing people. Here is a look of some of the expectations and responsibilities for this position:
- Work closely with data analysts and engineers to clean, transform, and migrate vast amounts of raw data from various systems using Spark into a Hadoop (Hortonworks) environment
- Prepare data for both ad-hoc requests as well as data pipelines that are embedded in our production environment
- Design and development using Spark, Hive, Python and other Hadoop tools
- Ensuring availability of needed data for selected reports
- Ensuring the correct data is available in the designated data repository
- Ensuring that there is an integration path between source and target data which includes multiple data sources.
- Works within an Agile Team
- Work with alternative analytic data systems to incorporate them into the operational data flow for the Analytics Teams. With leadership of engineering, work with data science teams and strategic partners on capabilities of core platform. This may include products purchased by the organization that must be ingested or modeled/derived data maintained by analytic teams.
- Keep current with Big Data technologies in order to recommend best tools in order to perform current and future work.
- Employ best-practices and standards with security and privacy in mind
- Bachelor's Degree in Computer Systems Analysis, Computer Engineering, Data Processing, Informatics or Management Information Systems
- Significant hands-on Hadoop development experience, preferably in the Hortonwork/Cloudera platform
- Proficient in Spark architecture with strong coding experience on Python.
- Experience in high volume data environments
- Experience on data lakes, datahub implementation
- Experience with the following technologies: Hadoop, Spark Hive, SQL and Python
- Experience working in an agile environment
- Full life-cycle experience (design through delivery and support)
- Data warehousing and/or Data Lake experience in Healthcare or Financial Services industry
- Outstanding troubleshooting/problem solving abilities.
- Inquisitive and resourceful– not afraid to network within the organizations to ask relevant questions
- Excellent verbal and written communication skills – ability to explain complex concepts to a wide variety of individuals with varying technical skills.
- Team builder and facilitator of cross team discussions with proven ability to identify and remove roadblocks and resolve issues
- Ability to handle sensitive and confidential information appropriately
This position is available as remote from home (Due to COVID-19 all work is currently remote).
At Techstra Solutions, we help top companies and brands achieve the business value of Digital and Talent Transformation. We believe there are three components in successful business transformation: Business Strategy, Technology and Talent. It is the coming together of these three disciplines that enable companies to take full advantage of opportunities. It differentiates us. Our approach is holistic and all encompassing. We consider the full picture as we guide our clients on this journey.
We are experts in transformation, business strategy, technology, innovation, and human capital management. We deliver our expertise through client consulting, innovative staffing solutions and software development. From strategy through implementation, we are dedicated to bringing our clients world-class business and talent solutions that fit strategic requirements and most importantly, deliver results.
Equal Employment Opportunity Statement
Techstra Solutions maintains a strong policy of equal opportunity in employment. It is our objective to recruit, hire, and retain the most qualified individuals without regard to race, color, creed, religion, gender, national origin, age, pregnancy, marital status, sexual orientation, gender identity or expression, disability, veteran status, or any other characteristic or status protected by applicable federal, state, or local law. Our equal employment philosophy applies to all aspects of employment, including recruitment, compensation, benefits, training, promotions, transfers, job benefits, and terminations.