Data Engineer - AWS

The Data Engineer is responsible for the maintenance, synchronization, cleaning, and migration of transactional data in a hybrid environment with both on-prem and highly modern cloud based microservices environment. The Data Engineer works with the product teams in order to understand, analyze, document and efficiently implement to deliver streaming as well as batch oriented data for synchronizing legacy and modern data stores ensuring data integrity.

The Data Engineer provides support to the Application database design to aid in eliminating data duplication and enabling selective event based and schedule based data transfer to endpoints within the cloud and legacy environment as required. The Data Engineer drives towards programmatic pipeline generation and orchestration to enhance repeatability and rapid deployment, using out of the box thinking, AWS native capabilities and CI/CD tools while utilizing established design patterns and methods.

Job Responsibilities:

• The successful candidate will be able to rapidly develop technical solutions working closely with the integrated product teams and developers with minimal direction from senior or lead resources
• Understand data needs and be able to construct data pipelines for automating event driven bi-directional selective data replication, along with micro-batch and batch based data pipelines
• Standardization of data processing modules to deliver modularity and enhance reusability
• Utilize identified tools and services such as AWS Glue, Python, Streamsets, Step functions and Lambda, Kinesis Data Streams and Data Firehose
• Create and maintain standards and best practices for data and pipeline standards

The candidate must have a successful track record in ETL job design, development and automation activities with minimal supervision. The candidate will be expected to support a variety of structured, semi-structured and unstructured data in streaming and batch frameworks. The candidate must possess AWS working knowledge and proven skills on AWS data tools and services such as AWS Glue, Step Functions, Lambda and DMS. Troubleshoot, monitor and coordinate defect resolution related to data processing & preparation; Responsible for the creation and support of all data pipeline processes across various data assets within the current scope of the system.

Required Skills:

• 7+ years of relevant work experience
• 3+ years of hands on experience in with ETL tools such as SAP DS and Pentaho PDI
• 4+ years of hands on experience in with python and various python toolkits and libraries for data processing and pipelines
• 4+ year of experience and ability to create complex SQL queries and functions
• 2+ years of hands on experience in AWS Glue, Python, Step Functions, Kinesis
• 1+ year of experience in java and Linux scripting is essential
• 1+ year experience working with CI/CD tools including Git for ETL and scripts repository

Desired Skills:

• AWS certifications are advantageous
• Experience working with SQL Server and tools like SSRS is advantageous
• Experience with big data tools such as EMR/Spark, Databricks/PySpark is an advantage
• Experience working with database versioning tools such as Flyway is an advantage
• Experience in Ansible and Jenkins scripting is an advantage

Education:

• Bachelor's degree in Computer Science or related discipline

T-Rex Overview

Established in 1999, T-Rex Solutions, LLC is a proven mid-tier business providing data-centric mission services to the Federal government as it increasingly tries to secure and leverage the power of data. We design, integrate, secure, and deploy advanced technical solutions for our customers so they can efficiently fulfill their critical objectives. T-Rex offers both IT and professional services to numerous Federal agencies and is a leader in providing high quality and innovative solutions in the areas of Cloud and Infrastructure Services, Cyber Security, and Big Data Engineering.

T-Rex is constantly seeking qualified people to join our growing team. We have built a broad client base through our devotion to delivering quality products and customer service, and to do that we need quality individuals. But more than that, we at T-Rex are committed to creating a culture that supports the development of every employee's personal and professional lives. T-Rex has made a commitment to maintain the status of an industry leader in compensation packages and benefits which includes competitive salaries, performance bonuses, training and educational reimbursement, Transamerica 401(k) and Cigna healthcare benefits.

T-Rex is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex (including pregnancy, gender identity, and sexual orientation), parental status, national origin, age, disability, family medical history or genetic information, political affiliation, military service, or other non-merit based factors.

As part of our overall strategy and commitment to maintaining a safe and healthy workplace, and in accordance with applicable regulations, T-Rex requires that all employees be fully vaccinated against COVID-19 as a condition of employment regardless of the employee's duty location or work arrangement (e.g., telework, remote work, etc.), subject to reasonable accommodation and other requirements under applicable federal, state, and local law. If selected, you will be required to submit documentation of proof of vaccination before starting employment with T-Rex or have an approved exemption prior to the start of your employment. Customer site vaccination requirements, if more strict, will take precedence over T-Rex's vaccination policy.

#ZR