Data Engineer

Posted 16 Days Ago
Be an Early Applicant
Glasgow, City of Glasgow, Scotland, GBR
Hybrid
Entry level
Agency • Digital Media • Professional Services • Design
The Role
Develop and support large-scale financial data workflows using Spark, Scala, Python, Airflow, SQL, and PostgreSQL. Build reusable data engineering components, optimize queries, troubleshoot production issues, and improve monitoring, automation, reconciliation, and operational resilience. Collaborate with application, platform, database, infrastructure, and downstream teams while following security, risk, testing, documentation, and change-control standards. The role is hybrid and requires BPSS eligibility.
Summary Generated by Built In
Company Description

Who is HelloKindred?

HelloKindred are specialists in staffing marketing, creative and technology roles, offering a range of talent solutions that can be delivered on-site, remotely or hybrid.

Our vision is to make work accessible and people’s lives better. We do this by disrupting traditional employment barriers – connecting ambitious talent to flexible opportunities with trusted brands.

Job Description

Anticipated Contract End Date/Length: December 31, 2026
Work set up: Hybrid (2–3 days per week in office)
Clearance required: BPSS checks. Candidates must be eligible for BPSS clearance

Our client in the Information Technology and Services industry is looking for a Data Engineer with strong Spark, Scala, and Python skills to develop, maintain, and troubleshoot data orchestration workflows and reusable engineering components. The role will support the processing, transformation, validation, and delivery of large-scale financial datasets, while contributing to platform engineering, production support, monitoring, automation, and operational resilience. The successful candidate will work across application, platform, database, infrastructure, and downstream teams to maintain reliable data processes and ensure compliance with technology, security, and change standards.

Responsibilities

  • Develop, maintain, and troubleshoot data orchestration workflows using Apache Airflow for extract generation, data publishing, feed delivery, and operational batch processes.
  • Build and support reusable engineering components using Python, Scala, and Sharedview Spark to process, transform, and validate large-scale financial datasets.
  • Design, write, and optimize SQL/PostgreSQL and Starburst/Trino queries for metadata processing, data validation, reconciliation, and operational investigation.
  • Support Output Gateway production issues, including failed DAGs, delayed extracts, missing files, data mismatches, checksum failures, logs, status updates, and downstream delivery exceptions.
  • Contribute to platform engineering and release activities using GitLab, Docker, and controlled CI/CD practices across DEV, UAT, and production-like environments.
  • Enhance monitoring, logging, alerting, audit tracking, and automation to improve operational stability, supportability, and evidence for control processes.
  • Work with application, platform, database, infrastructure, and downstream teams to diagnose issues end-to-end and implement safe, controlled fixes.
  • Maintain a strong risk and control mindset by ensuring changes are tested, reviewed, documented, and released in accordance with technology, security, and change standards.

Qualifications

  • Strong skills and experience in data engineering using Spark, Scala, and Python.
  • Experience developing, maintaining, and troubleshooting Apache Airflow workflows for data orchestration and operational batch processes.
  • Hands-on experience building reusable engineering components to process, transform, and validate large-scale financial datasets.
  • Proficiency in SQL and PostgreSQL, including query design, optimization, metadata processing, data validation, and reconciliation.
  • Experience supporting production data workflows and troubleshooting failed DAGs, delayed extracts, missing files, data mismatches, checksum failures, and downstream delivery exceptions.
  • Familiarity with GitLab, Docker, and controlled CI/CD practices across development, UAT, and production-like environments.
  • Understanding of monitoring, logging, alerting, audit tracking, and automation to improve operational stability and supportability.
  • Ability to collaborate with application, platform, database, infrastructure, and downstream teams to diagnose and resolve technical issues.
  • Strong awareness of risk, controls, testing, documentation, and technology, security, and change standards.

Desirable Skills / Preferred Qualifications

  • Experience with Starburst/Trino, AWS S3 concepts, and distributed query behavior for large-dataset validation and troubleshooting.
  • Exposure to React JS or API-driven operational user interfaces for configuration, monitoring, and support workflows.
  • Familiarity with OpenShift, Kubernetes, Helm, or container platform deployments in enterprise environments.
  • Understanding of financial data controls, auditability, reconciliation, data lineage, operational resilience, and data quality expectations.

Additional Information

All your information will be kept confidential according to EEO guidelines.

Candidates must be legally authorized to live and work in the country where the position is based, without requiring employer sponsorship.

HelloKindred is committed to fair, transparent, and inclusive hiring practices. We assess candidates based on skills, experience, and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully, only candidates selected for an interview will be contacted.

HelloKindred is an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race, colour, religion, sex, gender identity or expression, sexual orientation, age, national origin, disability, veteran status, or any other protected characteristic under applicable law.

Skills Required

  • Strong data engineering skills and experience using Spark, Scala, and Python
  • Experience developing, maintaining, and troubleshooting Apache Airflow workflows
  • Experience building reusable components for processing, transforming, and validating large-scale financial datasets
  • Proficiency in SQL and PostgreSQL, including query design, optimization, metadata processing, validation, and reconciliation
  • Experience supporting production data workflows and troubleshooting delivery and data-quality issues
  • Familiarity with GitLab, Docker, and controlled CI/CD practices
  • Understanding of monitoring, logging, alerting, audit tracking, and automation
  • Ability to collaborate across application, platform, database, infrastructure, and downstream teams
  • Awareness of risk controls, testing, documentation, and technology, security, and change standards
  • Experience with Starburst/Trino, AWS S3 concepts, and distributed query behavior
  • Exposure to React JS or API-driven operational user interfaces
  • Familiarity with OpenShift, Kubernetes, Helm, or container platform deployments
  • Understanding of financial data controls, auditability, reconciliation, data lineage, resilience, and data quality
  • Eligibility for BPSS checks
  • Legal authorization to work without employer sponsorship
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
300 Employees
Year Founded: 1998

What We Do

HelloKindred is a talent recruitment and staffing agency that connects companies with marketing, creative, and digital talent for short and long-term contracts.

Similar Jobs

Capco Logo Capco

Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Hybrid
Edinburgh, City of Edinburgh, Scotland, GBR
6000 Employees

Capco Logo Capco

Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Hybrid
Glasgow, City of Glasgow, Scotland, GBR
6000 Employees
Hybrid
Edinburgh, City of Edinburgh, Scotland, GBR
3693 Employees

Peterson Company Logo Peterson Company

Data Engineer

Utilities • Industrial • Manufacturing
In-Office
Aberdeen, City of Aberdeen, Scotland, GBR

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
Cravath, Swaine & Moore LLP Thumbnail
Legal Tech • Other • Professional Services
New York, NY
1200 Employees
VCA Animal Hospitals Thumbnail
Pet • Professional Services
Los Angeles, CA
35000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account