We are looking for a Data Engineer to join our growing team. You will be responsible for expanding and optimizing our data, data pipeline architecture and optimizing data flow across functional teams. The Data Engineer will work with our software developers, database architects, data analysts and data scientists on data initiatives Your Role and Responsibilities
Required Technical and Professional Expertise
- Create and maintain optimal data pipeline architecture.
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability,
Preferred Technical and Professional Expertise
- Programming experience and hands-on experience in building modern data platforms/pipelines
- Programming languages: e.g. Python, Scala, Java
- Frameworks: Spark, Hadoop, Hive (Beam, Pig, Kafka, Storm would be considered as a plus)
- Knowledge of: CI/CD, Apache Airflow, Oozie, orchestration tools
- Search engines: Elasticsearch, Solr
- Databases: NoSQL, Relational databases (columnar & row-oriented), Graph databases
- Experience in designing data pipelines, ETL, ELT
- Knowledge of Cloud platforms: e.g. AWS, Azure, GCP and their data services
- Interest in or experience with machine learning and artificial intelligence is a plus
- Tools: Jupyter Notebooks, Apache Zeppelin, BI Tools
R - alternative to Python for ML applications and Data Science
To be eligible for this role, you must be a British National with a minimum of 5 years unbroken UK residency, preferably 10 years.
About Business Unit
The Client Innovation Centre (CIC) is an innovative and exciting part of IBM. A fast-paced and dynamic business built to deliver high-value technical services to world renowned clients. All of our employees work in a consultancy environment on a variety of exciting projects across a range of sectors. Every project has its own unique working environment - this enables our employees to build their experience, skillset and network. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems?
Your Life @ IBM
We are proud of our training and development opportunities. As part of the CIC, you will be welcome to undertake a multitude of training courses with an extensive range of classroom, e-learning, interactive and video courses. With our support, you will have the opportunity to acquire industry recognised qualifications, gain valuable experience with globally renowned companies and work with industry leading technologies. This will enable you to grow within our business which we support via promotion and progression opportunities.
IBM’s greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.
Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 50 companies relying on the IBM Cloud to run their business.
At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.
IBM wants you to bring your whole self to work and for you this might mean the ability to work flexibly. If you are interested in a flexible working pattern, please talk to our recruitment team to find out if this is possible in the current working environment.
Being You @ IBM
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.