Job description
About the vacancy
Our client is one of the world’s top 10 investment companies with headquarters in Great Britain. They are building a data market that will aggregate and process data from multiple data feeds and systems like stock exchanges, news feeds, stock exchange brokers, internal quantitative systems and others. The project involves around 30 various specialists (30 Big Data engineers, 10 data and business analysts, InterSystems cache developers, Java developers and Quantitative analysts/developers, etc.).
As a big data engineer, you will be developing a platform that models and predicts trends in the financial markets.
Must have
Experience working with Apache NiFi or Hadoop (HDFS, MapReduce, HBase)
Hands-on experience with Shell scripting
Strong experience with SQL
Spoken English
Would be a plus
Experience working in an agile environment and understanding of various phases of the software development life cycle
Hands-on experience with Spark 2.0 or greater (+ Scala)
Experience working with Hive And Impala
Experience with Data modelling
Skills working with Oozie
Skills working with Airflow
Skills working with Datasets and Data frames
Skills using source code and version control systems like SVN, Git, etc.
Experience on CDP(Cloudera Data Platform)
Knowledge of Hadoop architecture components
Learn more about our policy of equal opportunities in employment
