Data Engineer for Pune & Hyderabad

Company Overview

Optical Arc Pvt. Ltd. is a full-service company of custom software development, mobile application development, database management systems, data visualization, ALM, PLM, and website development. With a focus on helping businesses grow and succeed, we offer end-to-end solutions tailored to our clients' unique needs. Join us on LinkedIn for industry insights, company updates, and career opportunities.

Location: Pune / Hyd (hybrid two days a week)

• Minimum 5+ years of experience.
• Scala development and design using Scala 2.10+ or Java development and design using Java 1.8+.
• Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, PySpark, Spark streaming, YARN, Kafka, Hive, Python, ETL frameworks, Map Reduce, SQL, RESTful services).
• Sound knowledge on working Unix/Linux Platform
• Hands-on experience building data pipelines using Hadoop components - Hive, Spark, Spark SQL,PySpark.
• Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible & Jenkins) and requirement management in JIRA.
• Understanding of big data modelling techniques using relational and non-relational techniques
• Experience on Debugging the Code issues and then publishing the highlighted differences to the development team/Architects;