Our Dublin based client is looking for an experienced AWS Senior Data Engineer to join their team. Exciting opportunity with a company who operate on a global scale.
Develop, build and maintain data pipeline architecture
Assemble large and complex data sets that meet various business requirements in a functional manner
Identify, create, and implement process improvements for internal use
Key member of the involved with building the infrastructure required for optimal extraction and transformation of data from a wide variety of sources using Python and AWS ‘big data’ technologies such as Glue, Lambda and EMR
Work closely with various stakeholders including the Product Managers, Executives, Engineers and Design teams to assist with technical issues and support the data infrastructure.
Work on various data tools for analytics and data scientist team members to help them in the process of building and optimizing the product into an world leader.
Strong Python coding knowledge
Previous experience creating/running Python or Pyspark projects working within AWS Glue.
Experience working with AWS Data Lakes
Previous work on AWS Data Pipeline and CI/CD processes
Technology skills with big data tools: Hadoop, Spark, Kafka, etc.
Working knowledge with data pipeline and workflow management tools: Luigi, Airflow, AWS
Experience with stream-processing systems: Kinesis, Spark-Streaming, etc
Familiarity with a variety of datasets, structured, semi structured and unstructured etc.
Experience building and optimizing ‘big data’ data pipelines, architecture and data sets.
Strong analytic skills
Experience and capacity to support and working with cross-functional teams in a highly dynamic and ever changing environment.
Essential and Desirable Skills
4-6+ years of total experience in a similar role
Must have a degree in Computer Science, Statistics, Informatics and related degree