Data Engineer

Skills Required

• Amazon EMR
• Elasticsearch
• Big Data
• Apache Airflow
• Amazon Redshift
• Plus 2 or more relevant data engineering skills
  
  

Job Description

Minfy Technologies is seeking a Senior Data Engineer with 8+ years of experience to join our dynamic team. As a Senior Data Engineer, you will play a crucial role in designing, building, optimizing, and operating cloud-based systems for data collection, storage, processing, packaging, and visualization. Your primary focus will be on extracting valuable insights from data, making it usable for advanced analytics, and supporting machine learning initiatives.

Key Responsibilities

• Identify data sources and data formats, and design data ingestion strategies using both batch and real-time techniques.
• Design and construct secure, scalable data repositories, with a focus on optimizing cost, data compression, scalability, and seamless integration with data processing tools.
• Architect data pipelines for data processing and transformation, employing ETL/ELT processes, SQL, Apache Spark, and other relevant tools to handle high-volume and high-velocity data.
• Utilize visualization tools to create meaningful charts and dashboards based on user-defined metrics, providing actionable insights to stakeholders.
• Collaborate with cross-functional teams to understand business requirements and develop data solutions that meet the organization's needs.
• Ensure data quality, integrity, and reliability throughout the data lifecycle.
• Stay updated with emerging technologies and best practices in data engineering and data analytics.
• Mentor junior team members and contribute to the overall growth and success of the data engineering team.
  
  

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer with a track record of designing and implementing data solutions in a cloud environment.
• Proficiency in Amazon EMR, Elasticsearch, Big Data technologies, Apache Airflow, and Amazon Redshift.
• Strong knowledge of data modeling, database design, and data warehousing concepts.
• Experience with both batch and real-time data processing.
• Solid programming skills, particularly in SQL and Python.
• Familiarity with data visualization tools (e.g., Tableau, Power BI) is a plus.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills, with the ability to work effectively in a team environment.
  
  

Responsibilities

Eligibilities: