Job description

You Will Need 4+ Years Of Experience In Developing Big Data Solutions, Including Experience With The Following

We are looking for a Big Data Engineer with a successful track record for building scalable data solutions in a fast-paced environment.
  • Developing Hadoop systems.
  • Loading disparate data sets and conducting pre-processing services using Hive or Pig.
  • Finalizing the scope of the system and delivering Big Data solutions.
  • Design, develop, document, and architect Hadoop applications
  • Develop MapReduce coding that works seamlessly on Hadoop clusters
  • Have working knowledge of SQL, NoSQL, data warehousing, and DBA
  • Complete knowledge of the Hadoop ecosystem and Hadoop Common
  • Solid Experience in ingesting the data from multiple data sources through APIs into HDFS.
  • Solid Experience in creating data pipelines using NiFi
  • Solid Experience dealing with different data types [structured, unstructured, semi structured] along with solid experience in dealing with different file formats [Text, Json, Avro, Parquet, ORC, etc]
  • Solid experience in creating transformation using Spark, Hive on Tez and different components existing in the Hadoop eco system
  • Proven Experience with Hive, Impala, NoSQL data bases querying.
  • Experience on Data Modelling Star, Snowflakes. Able to apply Kimball and Inmon Methodology
  • Experience with NoSQL, Hive, Impala.
  • Experience in Tuning bigdata transformations using Apache Spark and Hive QL.
  • Solid experience in Python, Java, Scala and SQL (including Spark SQL)
  • Solid experience in creating multithreading Python, Java applications to ingest data.
  • Solid experience in UNIX Bash Commands and Scripts.
  • Experience with databases like Postgres, Greenplum, MYSQL, and Oracle.
  • Experience in creating batch, real-time and near-real-time data pipelines.
  • Experience with streaming data technologies using [ Kafka, Spark Streaming, Apache Flink, etc.]
  • Align the organization?s big data solutions with their client initiatives as requested.
  • Work with domain experts to put together a delivery plan with and stay on track.
  • Utilize Big Data technologies to design, develop, and evolve scalable and fault-tolerant distributed components.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
March 30, 2023
March 30, 2023

Big Data Engineer

March 30, 2023

Big Data Engineer