Lead Data Engineer

Posted Yesterday
Hiring Remotely in United States
Remote or Hybrid
90K-140K Annually
Senior level
Information Technology • Database • Consulting
The Role
Lead technical design and implementation of cloud data pipelines and data warehouse models using PySpark, Snowflake, and AWS. Drive cloud migration, establish engineering standards, optimize performance, coordinate onshore/offshore teams, engage stakeholders, mentor junior engineers, and ensure delivery of insurance analytics solutions.
Summary Generated by Built In

We are seeking an experienced Lead Data Engineer to support complex data engineering initiatives within our insurance data and analytics practice. This role combines deep technical expertise with strong coordination skills, working closely with onshore and offshore teams, business stakeholders, and project leadership to deliver enterprise data modernization and migration programs. The candidate will serve as a technical point of contact for cross-functional teams while remaining hands-on with cloud data technologies.

Base Compensation Range: 90,000 – 140,000

The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process.

For more information on benefits and what we offer please visit us at https://www.exlservice.com/us-careers-and-benefits

Responsibilities

Technical Delivery

  • Design and implement end-to-end data pipelines using PySpark, Snowflake, and AWS cloud services
  • Architect scalable ELT/ETL workflows and data warehouse models supporting insurance analytics use cases
  • Drive data migration and modernization efforts from legacy environments to cloud-native platforms
  • Develop and review complex SQL transformations, stored procedures, and data quality validation frameworks
  • Establish and enforce data engineering standards, coding best practices, and pipeline documentation
  • Provide hands-on troubleshooting and performance optimization across the data stack

Team Coordination & Stakeholder Engagement

  • Coordinate day-to-day activities across onshore and offshore data engineering teams to ensure timely delivery
  • Serve as a technical point of contact for business stakeholders, translating requirements into engineering deliverables
  • Facilitate requirement-gathering sessions, sprint planning, and status updates with project teams
  • Communicate project progress, risks, and dependencies to project managers and client stakeholders
  • Mentor junior engineers and conduct code reviews to uphold quality standards
  • Collaborate with data architects, analysts, and QA teams throughout the project lifecycle

Required Skills & Qualifications

Technical Skills

  • Deep experience with Snowflake including data modeling, performance tuning
  • Proficiency with AWS services — S3, Glue, Lambda, EMR, Redshift, Step Functions, CloudWatch
  • Strong experience building distributed data processing frameworks with Apache Spark / PySpark
  • Advanced SQL skills — complex transformations, query optimization, and dimensional modeling
  • Expertise in DWH design patterns — Kimball, Inmon, Data Vault, star and snowflake schemas
  • Demonstrated experience leading or contributing to cloud migration and legacy modernization programs
  • Familiarity with tools such as dbt, Apache Airflow, AWS Glue, or similar orchestration frameworks
  • Solid Python programming for data engineering and automation tasks

Experience Requirements

  • 6–9 years of progressive experience in data engineering
  • Prior experience in insurance, financial services, or regulated industries preferred
  • Experience coordinating distributed teams across time zones (onshore/offshore model)
  • Demonstrated ability to engage with non-technical stakeholders and translate business requirements

Exposure to Agile/Scrum delivery methodology

Qualifications

 

  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field

Skills Required

  • Deep experience with Snowflake including data modeling and performance tuning
  • Proficiency with AWS services (S3, Glue, Lambda, EMR, Redshift, Step Functions, CloudWatch)
  • Strong experience building distributed data processing frameworks with Apache Spark / PySpark
  • Advanced SQL skills including complex transformations and query optimization
  • Expertise in data warehouse design patterns (Kimball, Inmon, Data Vault, star and snowflake schemas)
  • Experience leading or contributing to cloud migration and legacy modernization programs
  • Familiarity with dbt, Apache Airflow, AWS Glue, or similar orchestration frameworks
  • Solid Python programming for data engineering and automation tasks
  • 6-9 years of progressive experience in data engineering
  • Experience coordinating distributed onshore/offshore teams across time zones
  • Demonstrated ability to engage non-technical stakeholders and translate business requirements
  • Exposure to Agile/Scrum delivery methodology
  • Bachelor's degree in Computer Science, Information Systems, Engineering, or related field
  • Prior experience in insurance, financial services, or regulated industries
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
30,246 Employees
Year Founded: 1999

What We Do

Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.

Similar Jobs

In-Office or Remote
Deerfield, IL, USA
10000 Employees
161K-193K Annually

Beyond Finance Logo Beyond Finance

Platform Engineer

Fintech • Financial Services
Easy Apply
Remote or Hybrid
United States
2200 Employees
170K-205K Annually

Brillio Logo Brillio

Lead Data Engineer

Information Technology
In-Office or Remote
Seattle, WA, USA
2676 Employees
120-125 Annually

Photon Logo Photon

Lead Data Engineer

Agency • Information Technology
Remote
United States
5017 Employees
33K-118K Annually

Similar Companies Hiring

Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees
Standard Template Labs Thumbnail
Artificial Intelligence • Information Technology • Software
New York, NY
25 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account