Data Engineer ( Pyspark , Python & AWS )

Detailed JD *(Roles and Responsibilities)

• Understanding of core Business Intelligence BI data warehousing technology ETL, RDBMS etc.
• Work with the application development teams throughout the development life cycle.
• Create Airflow DAG pipelines for data ingestion.
• Hands on knowledge of Scripting languages like Python, PySpark, Spark SQL, Data Frames etc.
• Experience in AWS cloud services like S3, GLUE, REDSHIFT, LAMBDA, IAM, EMR, ATHENA, SPECTRUM.
• Expert in quickly understanding business needs and requirement gathering.
• Ability to take initiative and ownership on tasks.
• Good written and oral communication.
• SQL knowledge and performance tuning experience needed.
  
  

Minimum 5 to 6 years of experience and having done a lead experience with at least 4 members in the team.

Mandatory skills*

Python, Pyspark and AWS

Desired skills*

Python, Pyspark and AWS