Data Engineer

Job SummaryWe are looking for a Data Engineer (full-time, salary, regular) to join our team. The BI data engineering team is responsible for collecting, analyzing and distributing data using public cloud and open source technologies and offers transparency into customer behavior and business performance

ResponsibilitiesCollaborate with product teams and data analysts to design and build data-forward solutionsBuild and deploy streaming and batch data pipelines capable of processing and storing petabytes of data quickly and reliablyIntegrate with a variety of data metric providers ranging from advertising, web analytics, and consumer devicesBuild and maintain dimensional data warehouses in support of business intelligence toolsDevelop data catalogs and data validations to ensure clarity and correctness of key business metricsDrive and maintain a culture of quality, innovation and experimentationDeliver strong Python and SQL development and maintenance techniques surrounding data movement to include technologiesInvestigate and understand different data sources and ability to connect to a wide variety of 3rd party APIsDesign, enhance and implement ETL/data ingestion platform on the cloudDevelopment of ETL source and target mapping design/specifications based on the business requirements. Create ETLs/ELTs to take data from various operational systems and create a unified/enterprise data model for analytics and reportingDevelop load and transformation processes in support of the requirements, validate that they meet business and technical specifications, manage ongoing maintenance of the system and data, and make recommendations for process improvements to optimize data movement from source to targetProvide production and operational support to existing ETL jobs. Monitor and manage production ETL jobs to verify execution and measure performance to assure ongoing data quality and optimization of the system to manage scalability and performance and identify improvement opportunities for key ETL processesStrong troubleshooting and problem-solving skills in large data environmentCapable of investigating, familiarizing and mastering new data sets quickly

Supervisory ResponsibilityThis position will not include supervising one or more employees where applicable.

Education/ExperienceBachelor’s degree – Computer Science or equivalentStrong background in scripting language using Python, Bash, Perl, PHP or any other language to solving data problems ( At least worked on any one)Experience with relational SQL and NoSQL databases, including Postgres, ,Neo4j and MongoDbExperience with Big Data tools; Hadoop, Spark, Kafka, Hive etc ( At least worked on any one)Proficiency with data exchange types and protocols (json, xml, soap, rest)Experience with Stream Processing systems: Apache Storm ,Spark-Streaming etc ( At least worked on any one)Experience with BI tools like Tableau or any other open source BI tools etcProficiency with the AWS cloud services : EC2, EMR, RDS, S3, Redshift (spectrum)Exposure to cloud platforms (preferably AWS)

Knowledge, Skills, & AbilitiesKnowledge of the Python data ecosystem using pandas and numpyData integration toolsProficiency in SQL, data modeling, and data warehousingExcellent problem solving skillsExposure to cloud platforms (preferably AWS)

Equipment/Software UsedMicrosoft Office Suite (Outlook, Word, Excel, PowerPoint)SQL, MySQL or other relational databasesLinux, Python, AWS Stack (EC2,EMR S3, Redshift)Tableau or any other data visualization toolSiteCatalyst (Omniture)/Google Analytics or any other web analytics tools experience (Nice to have)