Big Data Engineer

Job description


The position that we are hiring will form part of a core Product Engineering team that is building a next generation big data analytics platform for the healthcare space. Due to the strategic nature and long-term vision of the product, candidates are expected to demonstrate steadfast commitment and dedication over and above proving themselves to excel in the challenging technical skillset.

Designs, develops, troubleshoots and debugs software programs and/or cloud-based applications on proprietary products and platforms for enhancements and new products in accordance with the needs of the organization.

Modifies software enhancements and/or new products used in local, networked, cloud-based or Internet-related computer programs.

Using current programming language and technologies, writes code, completes programming and performs testing and debugging of applications.

Ensures continuous, high velocity delivery and automated deployment using software provisioning, configuration management, source code management and team collaboration complementing the efficiencies of Agile software development methods.

Technical Skillset
  • Fluent in big data engineering development using the Hadoop/Spark ecosystem
  • Hands-on project experience with Cloudera Data Platform
  • Data ingestion and integration into the Data Lake using the Hadoop ecosystem tools such as Sqoop, PySpark, Impala, Hive, Oozie, Airflow etc.
  • Candidates should be fluent in the Python language
  • Developing the Data ingestion and integration flows in Hive, Spark and Impala
  • Creating the Hive Data structure, metadata and loading the data into Data Lake/Big Data warehouse environment
  • Building the data pipeline to migrate and load the data into the Hadoop distributed file system either on-prem or in the cloud
  • Experience in building real-time data ingestion pipelines using Apache Kafka and Apache NiFi
  • Developing applications with Apache Kudu and experience in Kudu integration with Spark
Non-technical Skills
  • Good English communication skills
  • Self-driven and self-initiated
  • Team player
  • Candidates with experience in healthcare big data projects will be preferred
Work Experience
  • Overall 4-6 years of Data Engineering experience in Big Data
  • 2-3 years of hands-on experience in Hadoop ecosystem tools such as Sqoop, Hive, Hbase
  • Hands-on development experience in Spark framework– PySpark/Spark-Scala/Java
  • Bachelor of Engineering or Bachelor of Technology

As an integral part of the product engineering team, responsibilities will include but not limited to -
  • Understanding the requirements from the Functional Team
  • Developing the code that aligns to the technical design and coding standards
  • Ownership of the code and deployment into test, UAT and production
  • Conduct Peer-Code Reviews for early detection of defects and code quality
  • Troubleshooting and follow escalation procedures to resolve issues
IQVIA is a leading global provider of advanced analytics, technology solutions and clinical research services to the life sciences industry. We believe in pushing the boundaries of human science and data science to make the biggest impact possible – to help our customers create a healthier world. Learn more at

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.