Data Engineer
Inside IR35 Rate: £500.00 per day
Duration: 6 Months
Location: UK Remote, occasional travel to London.
Clearance: Active SC Clearance
The Opportunity
We're looking for an experienced SC Cleared Data Engineer to join a large-scale data platform programme responsible for collecting, processing, and analysing vast volumes of customer interaction data.
Working within a collaborative Agile team, you'll help build and maintain modern data products that support analytics, risk assessment, and operational decision-making. This role offers the opportunity to work with large-scale datasets, cloud-native technologies, and modern data engineering practices in a "you build it, you run it" environment.
Key Responsibilities
Design, build, and maintain scalable data pipelines and data products.
Process, validate, cleanse, and enrich large datasets from multiple sources.
Develop and productionise data solutions using Python, Spark, and AWS technologies.
Create and maintain Jupyter notebooks to support analysis and data engineering activities.
Collaborate with multidisciplinary teams to deliver data-driven solutions.
Support the publication, sharing, and consumption of data products.
Maintain and improve automated data processing workflows.
Champion testing, automation, reliability, and continuous improvement.Essential Skills & Experience
Strong commercial experience with:
Python
Spark
SQL
Database technologies
AWS
Experience building ETL and data transformation solutions.
Strong understanding of data engineering best practices.
Experience working with large-scale datasets.
Knowledge of data modelling, validation, and data quality processes.
Experience working within Agile delivery environments.
Familiarity with CI/CD and automated deployment practices.
Experience supporting and maintaining production data platforms.Desirable Skills
Scikit-Learn or Spark MLlib
AWS EMR
Apache Airflow
Jenkins
Hudi or other open table formats
Kibana
Grafana
Understanding of Data Science principlesIf you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website
