Data Engineer (Cloud, Python)

Salary: £50,000 - 50,000 per year

Requirements:

• Experience delivering production data pipelines end to end, including design, development, implementation and operational maintenance in a cloud environment. • At least two years experience writing production-quality Python to work with complex, messy or large datasets. • An undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development. • Strong understanding of data engineering best practices, including data modelling, transformation, validation and testing. • Experience collaborating within multidisciplinary teams. • Strong communication skills, including the ability to explain technical work clearly to non-technical audiences.

Responsibilities:

• Design, build and maintain scalable data pipelines to ingest, transform and process large, complex datasets. • Develop automated web-scraping and bulk data-acquisition workflows. • Implement data transformation, validation, testing and quality controls to make datasets reliable, reproducible and ready for analysis. • Build and maintain containerised data and analytics applications, including dashboards and data tools. • Implement and manage workflow orchestration and scheduling so data pipelines and jobs are observable, reliable and maintainable. • Apply Infrastructure as Code to support data pipelines and processing workflows on AWS. • Design and maintain systems for data version control and…