Senior Data Engineer (Databricks)

Posted One Month Ago
Be an Early Applicant
London, Greater London, England, GBR
Hybrid
Senior level
Information Technology • Database • Consulting
The Role
Design, build, deploy, and optimize scalable data pipelines and ETL/ELT processes using Azure Databricks and related Azure services. Develop data models, data quality frameworks, integration solutions, and technical documentation for insurance data transformation programs. Collaborate with stakeholders, lead technical delivery, and apply data governance, warehousing, distributed computing, Agile, and DevOps practices.
Summary Generated by Built In

EXL (NASDAQ: EXLS) is a global data and AI company that offers services and solutions to reinvent client business models, drive better outcomes and unlock growth with speed. Bringing together domain expertise with robust data, powerful analytics, cloud and AI to create agile, scalable solutions and execute complex operations for the world’s leading corporations.

EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect, creating value from data to ensure faster decision-making and transforming operating models. Key industries include Insurance, Healthcare, Banking and Financial Services, Media, and Retail, among others. 

Headquartered in New York, our team is over 60,000 strong, with more than 50 offices spanning six continents. For information, visit www.exlservice.com.

Role Title:  Databricks Data Engineer

BU/Segment: Analytics / Data Management / Insurance Domain

Location:   London, United Kingdom (Hybrid, 3-4 days in the office)

Employment Type: Permanent to start ASAP

Summary of the role:

We’re looking for a highly skilled Azure Databricks Data Engineer to design, develop and deliver cloud-based data integration and analytics solutions within our Insurance portfolio.

You’ll work across data engineering, modelling, and transformation initiatives — building scalable pipelines, optimising performance, and ensuring best practices in data quality and governance. This is a hands-on technical role requiring strong Azure and Databricks experience, coupled with solid Insurance domain knowledge.

As part of this role, you’ll have the opportunity to work on high-impact Insurance data transformation programmes, collaborating with experienced data professionals in a forward-thinking, innovative environment. You’ll gain exposure to cutting-edge Azure and Databricks technologies.

As part of your duties, you will be responsible for:

  • Design, build, and deploy scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Database.
  • Optimise Databricks environments for performance, cost, and security.
  • Develop and maintain data integration, ETL/ELT processes, and data quality frameworks.
  • Translate business requirements into data models, ensuring consistency and accuracy across data layers.
  • Work with stakeholders to deliver effective data solutions, analytics.
  • Create technical documentation including data flow diagrams, architecture design, and test plans.
  • Stay current with emerging data engineering and Azure ecosystem technologies to drive continuous improvement.

Qualifications and experience we consider to be essential for the role:

  • Insurance industry experience is essential, with a strong understanding of policy, claims, and regulatory data.

  • Minimum of 5 years’ management experience required – leading technical delivery or overseeing project teams within data engineering or integration environments.

  • 8–10 years’ experience in data engineering and integration, ideally in enterprise-scale environments.

  • Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake.

  • Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design, and data pipeline orchestration.

  • Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts.

  • Experience working with large, complex datasets and distributed computing environments.

  • Proficiency in SQL, version control (e.g. Git), and CI/CD pipelines.

  • Familiarity with Agile delivery and DevOps practices (Azure DevOps, JIRA, etc.).

 

Preferred Skills & Experience:

 

  • Knowledge of EDM (Enterprise Data Management) practices, data governance, metadata management, and lineage tools.

  • Exposure to multi-cloud environments (AWS or GCP).
  • Understanding of modern data architecture philosophies (Dimensional, ODS, Data Vault) and real-time cloud data capabilities.

As part of a leading global analytics and AI solutions company, you can look forward to:

  • A competitive salary with a generous bonus, private healthcare, critical illness life assurance at 4 x your annual salary, income protection insurance, and a rewarding pension.
  • EXL provides everyday financial well-being solutions, such as cash back cards, in which you can earn cashback while enjoying discounts, promotions, and offers from top retailers. We also offer a Cycle Scheme where you can save money on bikes and cycling accessories.
  • At EXL, we are committed to providing our employees with the tools and resources they need to succeed and excel in their careers. We offer a wide range of professional and personal development opportunities. We also support a range of learning initiatives that allow our employees to build on their existing skills and knowledge. From online courses to seminars and workshops, our employees have the opportunity to enhance their skills and stay up to date with the latest trends and technologies.
  • As an Equal Opportunity Employer, EXL is committed to diversity. Our company does not discriminate based on race, religion, colour, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, age, or disability status.
  • EXL employees are eligible to purchase stock as part of our Employee Stock Purchase Plan (ESPP).
  • At EXL, we offer a flexible hybrid working model that allows employees to live a balanced, healthy lifestyle while strengthening our culture of collaboration.

To be considered for this role, you must already be eligible to work in the United Kingdom.


Skills Required

  • Insurance industry experience, including knowledge of policy, claims, and regulatory data
  • At least 5 years of management experience leading technical delivery or overseeing project teams in data engineering or integration
  • 8-10 years of experience in data engineering and integration
  • Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake
  • Hands-on experience with Spark, PySpark, SparkSQL, ETL/ELT design, and data pipeline orchestration
  • Strong data modeling skills using Dimensional, ODS, and Data Vault approaches
  • Experience with data warehousing concepts
  • Experience working with large, complex datasets and distributed computing environments
  • Proficiency in SQL, version control such as Git, and CI/CD pipelines
  • Familiarity with Agile delivery and DevOps practices, including Azure DevOps or JIRA
  • Knowledge of enterprise data management, data governance, metadata management, and lineage tools
  • Exposure to multi-cloud environments using AWS or GCP
  • Understanding of modern data architecture and real-time cloud data capabilities
  • Eligibility to work in the United Kingdom
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
30,246 Employees
Year Founded: 1999

What We Do

Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.

Similar Jobs

JPMorganChase Logo JPMorganChase

Data Engineer

Financial Services
Hybrid
London, Greater London, England, GBR
289097 Employees
In-Office
London, Greater London, England, GBR
289 Employees

Capco Logo Capco

Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Hybrid
London, England, GBR
6000 Employees

Similar Companies Hiring

Axle Health Thumbnail
Artificial Intelligence • Healthtech • Information Technology • Logistics
Santa Monica, CA
25 Employees
NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity
Sydney, AU
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account