We are looking for a proactive and hands-on Azure Data Engineer to join our engineering team. The successful candidate will be responsible for designing, developing, and supporting scalable data solutions using Microsoft Azure.
You will work closely with engineering, data, and delivery teams to build reliable data pipelines and cloud-based data platforms, while following best practices around infrastructure, automation, and source control.
- Design, develop, and maintain scalable Azure data solutions and pipelines.
- Build and manage data pipelines using Azure Data Factory (ADF).
- Develop and optimise data solutions using Azure Synapse Analytics.
- Work with Azure Data Lake for data storage and processing.
- Implement and maintain cloud infrastructure using Terraform.
- Use GitLab for source control, version management, and CI/CD processes.
- Monitor, troubleshoot, and optimise data pipelines and Azure services.
- Ensure data solutions are secure, scalable, reliable, and maintainable.
- Collaborate with developers, data analysts, architects, and delivery teams.
- Proactively identify technical issues and opportunities for improvement.
- Contribute to automation and continuous improvement of engineering practices.
- Strong experience working with Microsoft Azure cloud services.
- Hands-on experience with Azure Data Factory (ADF).
- Strong experience with Azure Synapse Analytics.
- Experience working with Azure Data Lake.
- Hands-on experience with Terraform and Infrastructure as Code (IaC).
- Experience using GitLab, including GitLab CI/CD.
- Strong data engineering and data pipeline development experience.
- Good understanding of cloud-based data architecture and integration.
- Strong troubleshooting and problem-solving skills.
- Proactive approach with the ability to work independently and as part of a team.
- Good communication and collaboration skills.
- Experience with Databricks.
- Experience with Apache Kafka.
- Experience with Kafka Connect.
- Knowledge of event-driven data architectures and real-time data processing.
Azure | Azure Data Factory | Azure Synapse Analytics | Azure Data Lake | Terraform | GitLab | Databricks | Kafka | Kafka Connect
We welcome applications from candidates of all backgrounds and actively encourage individuals from underrepresented groups, including women, people with disabilities, and diverse communities, as well as those seeking flexible working arrangements.
As a Disability Confident Level 1 employer, we are committed to providing reasonable adjustments throughout the recruitment process and during employment to ensure equal opportunities for all.
Skills Required
- Strong experience working with Microsoft Azure cloud services
- Hands-on experience with Azure Data Factory
- Strong experience with Azure Synapse Analytics
- Experience working with Azure Data Lake
- Hands-on experience with Terraform and Infrastructure as Code
- Experience using GitLab, including GitLab CI/CD
- Strong data engineering and data pipeline development experience
- Understanding of cloud-based data architecture and integration
- Strong troubleshooting and problem-solving skills
- Ability to work independently and as part of a team
- Good communication and collaboration skills
- Experience with Databricks
- Experience with Apache Kafka
- Experience with Kafka Connect
- Knowledge of event-driven data architectures and real-time data processing
What We Do
Scrumconnect Consulting is a UK national SME and award-winning software development consultancy with more than 300 consultants across the UK. It works with public-sector clients to build impactful digital services that improve millions of lives, combining design, engineering, and AI expertise. The company emphasizes experienced consultants, collaboration, knowledge-sharing, problem-solving, and a culture focused on sustaining innovation for its customers.
