May 25, 2023
Apply Now
United States, Remote
May 25, 2023
Apply NowJob description
Software Guidance & Assistance, Inc., (SGA), is searching for a Data Engineer for a FULL TIME assignment with one of our premier Technology clients in Fairfax, VA .
About the Role:
Our client is a market intelligence software company located in the Washington, D.C. area.
We have an existing commercial SaaS platform that consists of 3 components: a web application, several 3rd party databases integrated into our backend, and a Natural Language Processing, supervised ML model based on a custom taxonomy.
We are looking to build 2.0 of our platform, with a brand new front end based on new algorithms and an enhanced supervised deep learning model that uses a confluence of data from various data sources (e.g., patent, financial, and market data). It's a challenge and a fun opportunity for someone looking to make the next big platform that the world is going to use.
Our Data Engineer would have the opportunity to build strong Data and ETL infrastructure and practices to help enable this next generation of our machine learning. You will provide technical expertise and support in the design, development, implementation, and testing of 2.0. You will also participate in and/or direct major deliverables of projects through all aspects of the data lifecycle including the storage, processing and management of data; architecture and design; and building, managing and executing our ETL processes.
Our Current Stack: AWS to host the infrastructure, including the CICD, Java (SpringBoot)+ Angular, Python, Kubernetes, EMR, Spark, Elasticsearch, PostgreSQL, AWS (S3, Code Commit, Code Build, Code Deploy, EC2, EMR, etc.), Docker, Spacy, Scikit learn, Openpyxl, Streamlit, Watchdog, sklearn, seaborn, nltk, matplotlib, pandas, SQLAlchemy.
Primary Responsibilities:
Lead the ETL projects to support 2.0 platform development.
Using multiple technologies, design and implement scalable, reliable distributed data processing frameworks and analytical infrastructure.
Define, design, and implement data integration, management, storage, consumption, backup, and recovery solutions that ensure the high performance of the organization's enterprise data.
Integrate new data sources into the existing backend, ensuring interoperability and integrity with the current platform flows
Maintain data pipelines & data systems
Design, build, and automate solutions utilizing AWS services not limited to IAM, Glue, Lambda, S3, Athena, SNS, SQS, DynamoDB, RDS, EMR, ECS, Route 53 and Redshift.
Experience Required:
B.S. in relevant technical degree
Significant experience with scripting languages, such as Python, required
Significant use and experience with writing complex SQL queries, PostgreSQL required
Experience with AWS services (e.g., AWS Glue, AWS Lambda, S3, EC2, IAM, etc.) and AWS-native CI/CD tools (e.g., Code Build, Code Deploy, etc.)
3+ years of experience as a developer / engineer in an AWS cloud environment
Experience in designing data pipelines & data solutions for analytical data warehousing
Practical experience with Cloud methodologies (IaaS, PaaS, SaaS), automation, orchestration, trends and industry-leading cloud vendor offerings
Experience in building CI/CD pipeline and test automation in tools like Jenkins, GIT and/or AWS CodePipeline/CodeBuild/CodeDeploy
Experience building data-driven unit test suites for data platforms
Experience modelling highly dimensional data
Ability to work independently and integrate with other team members
Project management skills, to scope out timeline, methodology, and deliverables for development, testing, and integration into the platform
Excellent communication skills (written and verbal)
Well versed with using version control systems
Logistics: Geography, Work Status, Etc.
The position is remote. The candidate must have the legal right to work in the United States We are also looking for someone full-time to join us immediately.
About the Role:
Our client is a market intelligence software company located in the Washington, D.C. area.
We have an existing commercial SaaS platform that consists of 3 components: a web application, several 3rd party databases integrated into our backend, and a Natural Language Processing, supervised ML model based on a custom taxonomy.
We are looking to build 2.0 of our platform, with a brand new front end based on new algorithms and an enhanced supervised deep learning model that uses a confluence of data from various data sources (e.g., patent, financial, and market data). It's a challenge and a fun opportunity for someone looking to make the next big platform that the world is going to use.
Our Data Engineer would have the opportunity to build strong Data and ETL infrastructure and practices to help enable this next generation of our machine learning. You will provide technical expertise and support in the design, development, implementation, and testing of 2.0. You will also participate in and/or direct major deliverables of projects through all aspects of the data lifecycle including the storage, processing and management of data; architecture and design; and building, managing and executing our ETL processes.
Our Current Stack: AWS to host the infrastructure, including the CICD, Java (SpringBoot)+ Angular, Python, Kubernetes, EMR, Spark, Elasticsearch, PostgreSQL, AWS (S3, Code Commit, Code Build, Code Deploy, EC2, EMR, etc.), Docker, Spacy, Scikit learn, Openpyxl, Streamlit, Watchdog, sklearn, seaborn, nltk, matplotlib, pandas, SQLAlchemy.
Primary Responsibilities:
Lead the ETL projects to support 2.0 platform development.
Using multiple technologies, design and implement scalable, reliable distributed data processing frameworks and analytical infrastructure.
Define, design, and implement data integration, management, storage, consumption, backup, and recovery solutions that ensure the high performance of the organization's enterprise data.
Integrate new data sources into the existing backend, ensuring interoperability and integrity with the current platform flows
Maintain data pipelines & data systems
Design, build, and automate solutions utilizing AWS services not limited to IAM, Glue, Lambda, S3, Athena, SNS, SQS, DynamoDB, RDS, EMR, ECS, Route 53 and Redshift.
Experience Required:
B.S. in relevant technical degree
Significant experience with scripting languages, such as Python, required
Significant use and experience with writing complex SQL queries, PostgreSQL required
Experience with AWS services (e.g., AWS Glue, AWS Lambda, S3, EC2, IAM, etc.) and AWS-native CI/CD tools (e.g., Code Build, Code Deploy, etc.)
3+ years of experience as a developer / engineer in an AWS cloud environment
Experience in designing data pipelines & data solutions for analytical data warehousing
Practical experience with Cloud methodologies (IaaS, PaaS, SaaS), automation, orchestration, trends and industry-leading cloud vendor offerings
Experience in building CI/CD pipeline and test automation in tools like Jenkins, GIT and/or AWS CodePipeline/CodeBuild/CodeDeploy
Experience building data-driven unit test suites for data platforms
Experience modelling highly dimensional data
Ability to work independently and integrate with other team members
Project management skills, to scope out timeline, methodology, and deliverables for development, testing, and integration into the platform
Excellent communication skills (written and verbal)
Well versed with using version control systems
Logistics: Geography, Work Status, Etc.
The position is remote. The candidate must have the legal right to work in the United States We are also looking for someone full-time to join us immediately.
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at https://sgainc.com .
EEO Employer: Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status.
