Data Engineer

We are on the hunt for a Senior Data Engineer who is passionate about building large-scale data processing systems and has a knack for entrepreneurship. If you're someone who thrives in a fast-paced environment and loves working with financial data, then this could be the perfect role for you! In this role, you will lead the design and development of ETL pipelines and be responsible for owning the data lifecycle management, ensuring that data is properly classified, stored, and secured. You will work alongside data scientists and analysts to understand their data requirements and provide the necessary infrastructure to support their work. Salary - £70-90k Hybrid - London office 2x a week Essential Bachelor's, Master's or PhD in Computer Science, Data Science or related field 3+ years Data Engineer experience Python ETL AWS/AWS Glue/AWS Lambda TDD code reviews APIs CI/CD PostgreSQL Data Science toolkits: Jupyter Notebook, NumPy, Pandas, Scikit-learn etc Data modelling & Schema design Mentoring experience Entrepreneurial mindsetDesirable Docker/Kubernetes Familiarity with big data technologies Financial data experienceResponsibilities Design and implement large-scale data processing systems on AWS that are scalable, efficient, and reliable. Lead the development of ETL pipelines that transform raw data into usable data sets for analysis. Build APIs and data ingestion processes to facilitate seamless data access across the organization. Maintain engineering best practices and processes such as TDD, code reviews, code refactoring, branch management, and CI/CD practices to ensure code quality and reliability. Collaborate with other engineers on areas such as DevOps, infrastructure, and security to ensure that our data systems are properly integrated with the larger technology stack. Support technical lead with estimates and architectural decisions to ensure timely and regular delivery of features over time. Develop and maintain technical specifications, data flow diagrams, and architecture diagrams for data processing systems.Benefits Dedicated time to learn new skills Onsite gym Private health insurance Performance-based bonusIf this role sounds like a perfect match for you then we would love to hear from you! Get in touch and send over your CV today! Keywords: Data Engineer, Python, ETL, AWS, NumPy, Pandas, PostgreSQL BeTechnology Group Limited is acting as an Employment Agency in relation to this vacancy