Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time
We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.
The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.
- Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
- Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
- Define enterprise data models, ingestion frameworks, and transformation strategies.
- Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
- Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
- Drive cloud data platform modernization and migration initiatives.
- Serve as the technical lead for Databricks and Data Engineering programs.
- Establish engineering best practices, coding standards, and architecture guidelines.
- Conduct architecture reviews, code reviews, and solution assessments.
- Mentor and guide Data Engineers, Technical Leads, and junior team members.
- Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.
- Design cloud-native data solutions on Azure, AWS, or GCP.
- Implement secure, scalable, and resilient data architectures.
- Optimize costs, performance, reliability, and availability of data platforms.
- Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
- Define and implement monitoring, observability, and operational excellence practices.
- Establish data governance, lineage, cataloging, and security controls.
- Ensure compliance with organizational and regulatory requirements.
- Implement data quality frameworks and monitoring solutions.
- Collaborate with security and compliance teams to enforce data protection standards.
- Partner with business leaders to understand strategic data requirements.
- Translate business requirements into scalable technical solutions.
- Drive delivery planning, risk management, and resource coordination.
- Present architecture decisions and technical recommendations to leadership teams.
- Databricks Lakehouse Platform
- Databricks Workflows
- Delta Lake
- Unity Catalog
- Structured Streaming
- Databricks SQL
- Delta Live Tables (DLT)
- Databricks Asset Bundles
- Apache Spark
- PySpark
- Spark SQL
- Kafka
- Real-Time Streaming Architectures
- Data Lake & Lakehouse Architectures
- Azure Databricks
- Azure Data Factory (ADF)
- ADLS Gen2
- Azure Synapse Analytics
- Azure Key Vault
- Event Hub
- S3
- EMR
- Glue
- Redshift
- Lambda
- IAM
- BigQuery
- Cloud Storage
- Dataflow
- Pub/Sub
- Composer
- Python
- SQL
- PySpark
- Scala (Preferred)
- Shell Scripting
- Git
- Azure DevOps
- GitHub Actions
- Jenkins
- Terraform
- Docker
- Kubernetes
- Dimensional Modeling
- Star & Snowflake Schema
- Data Vault
- Master Data Management
- Data Governance Frameworks
- 12-15 years of overall IT experience.
- Minimum 6+ years of hands-on Databricks experience.
- 7+ years of experience in Spark/PySpark-based data engineering.
- Experience designing enterprise-scale Data Lake and Lakehouse solutions.
- Strong experience leading large data transformation and modernization programs.
- Experience handling structured, semi-structured, and unstructured data processing.
- Experience supporting analytics, AI/ML, and reporting ecosystems.
- Proven experience managing technical teams and driving architectural decisions.
- Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
- Databricks Certified Data Engineer Professional.
- Azure Data Engineer Associate.
- AWS Data Analytics Specialty.
- Google Professional Data Engineer Certification.
- TOGAF or Cloud Architecture certifications are a plus.
- MLOps and AI Engineering exposure.
- Snowflake and Databricks integration experience.
- Microsoft Fabric exposure.
- Data Mesh implementation experience.
- Data Governance tools such as Collibra, Alation, or Informatica.
- Expertise in Data Quality frameworks such as Great Expectations or Deequ.
- Solution Architecture
- Technical Leadership
- Stakeholder Management
- Program Delivery
- Data Strategy & Governance
- Team Mentoring & Coaching
- Problem Solving & Decision Making
- Communication & Executive Presentation Skills
- Performance Optimization
- Innovation & Continuous Improvement
Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.
Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time
We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.
The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.
- Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
- Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
- Define enterprise data models, ingestion frameworks, and transformation strategies.
- Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
- Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
- Drive cloud data platform modernization and migration initiatives.
- Serve as the technical lead for Databricks and Data Engineering programs.
- Establish engineering best practices, coding standards, and architecture guidelines.
- Conduct architecture reviews, code reviews, and solution assessments.
- Mentor and guide Data Engineers, Technical Leads, and junior team members.
- Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.
- Design cloud-native data solutions on Azure, AWS, or GCP.
- Implement secure, scalable, and resilient data architectures.
- Optimize costs, performance, reliability, and availability of data platforms.
- Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
- Define and implement monitoring, observability, and operational excellence practices.
- Establish data governance, lineage, cataloging, and security controls.
- Ensure compliance with organizational and regulatory requirements.
- Implement data quality frameworks and monitoring solutions.
- Collaborate with security and compliance teams to enforce data protection standards.
- Partner with business leaders to understand strategic data requirements.
- Translate business requirements into scalable technical solutions.
- Drive delivery planning, risk management, and resource coordination.
- Present architecture decisions and technical recommendations to leadership teams.
- Databricks Lakehouse Platform
- Databricks Workflows
- Delta Lake
- Unity Catalog
- Structured Streaming
- Databricks SQL
- Delta Live Tables (DLT)
- Databricks Asset Bundles
- Apache Spark
- PySpark
- Spark SQL
- Kafka
- Real-Time Streaming Architectures
- Data Lake & Lakehouse Architectures
- Azure Databricks
- Azure Data Factory (ADF)
- ADLS Gen2
- Azure Synapse Analytics
- Azure Key Vault
- Event Hub
- S3
- EMR
- Glue
- Redshift
- Lambda
- IAM
- BigQuery
- Cloud Storage
- Dataflow
- Pub/Sub
- Composer
- Python
- SQL
- PySpark
- Scala (Preferred)
- Shell Scripting
- Git
- Azure DevOps
- GitHub Actions
- Jenkins
- Terraform
- Docker
- Kubernetes
- Dimensional Modeling
- Star & Snowflake Schema
- Data Vault
- Master Data Management
- Data Governance Frameworks
- 12-15 years of overall IT experience.
- Minimum 6+ years of hands-on Databricks experience.
- 7+ years of experience in Spark/PySpark-based data engineering.
- Experience designing enterprise-scale Data Lake and Lakehouse solutions.
- Strong experience leading large data transformation and modernization programs.
- Experience handling structured, semi-structured, and unstructured data processing.
- Experience supporting analytics, AI/ML, and reporting ecosystems.
- Proven experience managing technical teams and driving architectural decisions.
- Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
- Databricks Certified Data Engineer Professional.
- Azure Data Engineer Associate.
- AWS Data Analytics Specialty.
- Google Professional Data Engineer Certification.
- TOGAF or Cloud Architecture certifications are a plus.
- MLOps and AI Engineering exposure.
- Snowflake and Databricks integration experience.
- Microsoft Fabric exposure.
- Data Mesh implementation experience.
- Data Governance tools such as Collibra, Alation, or Informatica.
- Expertise in Data Quality frameworks such as Great Expectations or Deequ.
- Solution Architecture
- Technical Leadership
- Stakeholder Management
- Program Delivery
- Data Strategy & Governance
- Team Mentoring & Coaching
- Problem Solving & Decision Making
- Communication & Executive Presentation Skills
- Performance Optimization
- Innovation & Continuous Improvement
Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.
Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time
We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.
The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.
- Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
- Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
- Define enterprise data models, ingestion frameworks, and transformation strategies.
- Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
- Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
- Drive cloud data platform modernization and migration initiatives.
- Serve as the technical lead for Databricks and Data Engineering programs.
- Establish engineering best practices, coding standards, and architecture guidelines.
- Conduct architecture reviews, code reviews, and solution assessments.
- Mentor and guide Data Engineers, Technical Leads, and junior team members.
- Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.
- Design cloud-native data solutions on Azure, AWS, or GCP.
- Implement secure, scalable, and resilient data architectures.
- Optimize costs, performance, reliability, and availability of data platforms.
- Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
- Define and implement monitoring, observability, and operational excellence practices.
- Establish data governance, lineage, cataloging, and security controls.
- Ensure compliance with organizational and regulatory requirements.
- Implement data quality frameworks and monitoring solutions.
- Collaborate with security and compliance teams to enforce data protection standards.
- Partner with business leaders to understand strategic data requirements.
- Translate business requirements into scalable technical solutions.
- Drive delivery planning, risk management, and resource coordination.
- Present architecture decisions and technical recommendations to leadership teams.
- Databricks Lakehouse Platform
- Databricks Workflows
- Delta Lake
- Unity Catalog
- Structured Streaming
- Databricks SQL
- Delta Live Tables (DLT)
- Databricks Asset Bundles
- Apache Spark
- PySpark
- Spark SQL
- Kafka
- Real-Time Streaming Architectures
- Data Lake & Lakehouse Architectures
- Azure Databricks
- Azure Data Factory (ADF)
- ADLS Gen2
- Azure Synapse Analytics
- Azure Key Vault
- Event Hub
- S3
- EMR
- Glue
- Redshift
- Lambda
- IAM
- BigQuery
- Cloud Storage
- Dataflow
- Pub/Sub
- Composer
- Python
- SQL
- PySpark
- Scala (Preferred)
- Shell Scripting
- Git
- Azure DevOps
- GitHub Actions
- Jenkins
- Terraform
- Docker
- Kubernetes
- Dimensional Modeling
- Star & Snowflake Schema
- Data Vault
- Master Data Management
- Data Governance Frameworks
- 12-15 years of overall IT experience.
- Minimum 6+ years of hands-on Databricks experience.
- 7+ years of experience in Spark/PySpark-based data engineering.
- Experience designing enterprise-scale Data Lake and Lakehouse solutions.
- Strong experience leading large data transformation and modernization programs.
- Experience handling structured, semi-structured, and unstructured data processing.
- Experience supporting analytics, AI/ML, and reporting ecosystems.
- Proven experience managing technical teams and driving architectural decisions.
- Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
- Databricks Certified Data Engineer Professional.
- Azure Data Engineer Associate.
- AWS Data Analytics Specialty.
- Google Professional Data Engineer Certification.
- TOGAF or Cloud Architecture certifications are a plus.
- MLOps and AI Engineering exposure.
- Snowflake and Databricks integration experience.
- Microsoft Fabric exposure.
- Data Mesh implementation experience.
- Data Governance tools such as Collibra, Alation, or Informatica.
- Expertise in Data Quality frameworks such as Great Expectations or Deequ.
- Solution Architecture
- Technical Leadership
- Stakeholder Management
- Program Delivery
- Data Strategy & Governance
- Team Mentoring & Coaching
- Problem Solving & Decision Making
- Communication & Executive Presentation Skills
- Performance Optimization
- Innovation & Continuous Improvement
Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.
Skills Required
- 12-15 years of overall IT experience
- Minimum 6 years of hands-on Databricks experience
- 7 or more years of Spark or PySpark data engineering experience
- Experience designing enterprise-scale data lake and Lakehouse solutions
- Experience leading large data transformation and modernization programs
- Experience processing structured, semi-structured, and unstructured data
- Experience supporting analytics, AI/ML, and reporting ecosystems
- Experience managing technical teams and driving architectural decisions
- Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field
- Databricks Certified Data Engineer Professional certification
- Azure Data Engineer Associate certification
- AWS Data Analytics Specialty certification
- Google Professional Data Engineer certification
- TOGAF or cloud architecture certification
- MLOps and AI Engineering exposure
- Snowflake and Databricks integration experience
- Microsoft Fabric exposure
- Data Mesh implementation experience
- Experience with Collibra, Alation, or Informatica
- Experience with Great Expectations or Deequ
EXL Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about EXL and has not been reviewed or approved by EXL.
-
Healthcare Strength — Medical plan options include HSA, HRA, and PPO choices, with additional health-related programs such as disability coverage and an employee assistance program. Feedback suggests these options provide meaningful flexibility and coverage breadth for many roles.
-
Leave & Time Off Breadth — Up to 20 days of paid time off, plus federal holidays and paid sick and safe time, are highlighted. Feedback suggests this leave structure is a notable benefit alongside core compensation.
-
Retirement Support — A traditional and Roth 401(k) with up to a 3% company match and full vesting after one year is described. Feedback suggests this match and vesting timeline help bolster long-term savings.
EXL Insights
What We Do
Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.

.jpeg)






