Big Data Engineer

SQL, Spark, Hive, AWS, Scala, EMR

Responsibilities:

Looking for a Big Data Engineer to assist with design, development, and testing of various database and ETL components within a cloud-based enterprise business application. This application handles the sales compensation function of a large telecom company, processing millions of transactions in each batch process.

The role is responsible for analyzing business requirements, and delivering database objects (SQL, Stored Procedures, ETL functions) to meet those requirements. Must be able to write efficient SQL Queries for the most optimal system performance.

The project follows an Agile development methodology.

Requirements:

• Proficient in writing complex SQL Queries and Stored Procedures in SQL Server
• Good knowledge of database design concepts for optimal query performance.
• Proficiency in Spark SQL and Scala in AWS environment
• Hands-on experience to write MapReduce jobs.
• Good knowledge of spark architecture, multi-threading, and concurrency concepts
• Good understanding of Hadoop, Hive, Hue, YARN, AWS EMR