Salary Dependent on Experience
Closing Date for this application is 30/06/2020
If this position receives high volumes of applications we reserve the right to close this advert earlier than stated, so please apply early to avoid disappointment.
We are seeking a Data Engineer to join us in our growing Data Infrastructure & Engineering team; to help develop, maintain, support, and integrate our growing number of data systems. The right candidate will have a wealth of knowledge in the data world and will be keen to expand upon their existing knowledge set learning new technologies along the way as well as supporting legacy technologies and processes.
In this role, you will:
- Ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple users and applications within the Group
- Help facilitate getting data from a variety of different sources, getting it into the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly.
- Enable big data and batch / real-time analytical solutions that leverage emerging technologies
- Develop prototypes and proof of concepts for selected solutions
- Implement complex big data projects with a focus on collecting, parsing, managing, analysing, and visualising large sets of data using multiple platforms
You would be responsible for:
- Coding, testing, and documenting new or modified data systems; creating scalable, repeatable, secure pipelines and applications for data analytics, both internally and externally to the business
- Growing the capabilities of the data platform(s), solving new data problems and challenges
- Supporting all manner of data flows for batch/real-time analytical solutions leveraging transformational technologies
- Maintaining the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly
- Ensuring that all automated processes preserve data by managing the alignment of data availability and integration processes
- Designing key and indexing schemas and designs partitioning
- Assisting with the development, testing, implementation, monitoring and maintenance of 24x7 production database management systems
- Participating in building and testing business continuity and disaster recovery procedures as per requirements
- Ensuring that all source data systems in the production environment remain stable and usable
Required education and experience:
- A minimum 5 years’ experience software development or database experience
- Proficiency in writing intermediate-level SQL including stored procedures in languages such as T-SQL. PL/PgSQL and PL/SQL
- Proficiency in one or more database scripting languages (i.e., PowerShell, Python etc.,)
- Experience with relational database internals, including both query processing and query planning, or other data processing infrastructure across several different RDBMS systems as well as NoSQL systems
- A basic knowledge of key data structures and algorithms
- Capability in programming in 3GL languages such as C#, C, C++ or Java
- A solid understanding of physical database design principles, and the system development life cycle.
- Understand how to apply technologies to solve data problems and to develop innovative data related solutions
- Knowledge of data analysis, end-user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solution
- Experience with object-oriented design, coding, and testing patterns
- Extensive experience in Microsoft SSIS, specifically designing and implementing ETL packages
- Knowledge of data modelling and understanding of different data structures and their benefits and limitations
- Familiarity with monitoring, backup, and disaster recovery of data systems
- Experience in designing, architecting, and building scalable pipelines
- Extensive experience with Linux, Unix and Microsoft platforms.
Desirable skills & Technologies
- Knowledge and usage of BI Tools such as Looker, Power BI, Tableau
- Experience with Spark and other analytical programming languages
- Experience in utilising Microsoft Azure and any related tooling for data engineering (such as Azure Data Warehouse, Azure Data Lake, EventHub, Apache Spark, Azure Database, Azure Databricks, Synapse)
- Experience on engineering big data systems
- A solid working knowledge of database administration methodologies and techniques
- A good understanding of logical and physical database design principles.
- Strong communication skills at all levels
- Be a natural problem solver
- Work well in both functional and cross-functional environments
- Exhibit flexibility when addressing daily shifting work priorities (we are a 24x7 business)
- Have the ability to work well under pressure, using own initiative as well as part of the wider organisation
If you believe you meet the above criteria and feel that this could be the role for you then APPLY NOW!