Manager

Posted 7 Hours Ago
Be an Early Applicant
Hyderabad, Telangana, IND
Hybrid
Expert/Leader
Information Technology • Database • Consulting
The Role
Lead the technical design and implementation of a Databricks lakehouse, including ingestion, data modeling, governance, security, pipelines, and performance optimization. Migrate legacy Hadoop, Hive, Spark, and warehouse workloads to Databricks while establishing engineering, testing, CI/CD, and observability standards. Mentor data engineers, lead design reviews, manage cost and security controls, and partner with analytics, ML, risk, fraud, and finance teams to deliver reliable data products.
Summary Generated by Built In

Role: Databricks Principal Engineer

Experience: 12+

Location: All EXL Location  

Work Mode: Hybrid 

POSITION SUMMARY


We are building our next-generation data platform on Databricks. The Principal Engineer sets the technical direction for how the bank ingests, models, governs and serves data. This is a hands-on role: you will design the detailed lakehouse implementation, set engineering standards for SQL and pipelines, lead the migration from legacy platforms, and mentor engineers across several teams. You will work within the target-state architecture owned by the Solution Architect.


ROLES AND RESPONSIBILITIES: 


  • Lakehouse design: Own the detailed design of the Databricks lakehouse: ingestion, bronze/silver/gold layers, Delta Lake design, Unity Catalog setup, and workspace and cluster configuration.
  • Migration: Lead the hands-on migration of legacy Hadoop/Hive/Spark and warehouse workloads to Databricks, including re-platforming HQL and SQL, validating data parity, and safe cutover.
  • Engineering standards: Set standards for SQL, PySpark and Delta Live Tables/Lakeflow pipelines, covering code review, testing, CI/CD and observability.
  • Performance and cost: Tune performance and cost: partitioning, liquid clustering/Z-ordering, file compaction, join and shuffle tuning, Photon, cluster right-sizing and DBU spend governance.
  • Data modeling: Design canonical, auditable entity models (applications, accounts, customers, transactions) across source systems.
  • Security and compliance: Implement PII protection, encryption and tokenization, row/column-level security, lineage, retention and audit readiness with Security, Risk, Compliance and Data Governance.
  • Design reviews: Lead technical design reviews, write architecture decision records, and resolve cross-team technical trade-offs.
  • Mentoring: Mentor senior and mid-level engineers and raise engineering quality across the data organization.
  • Data products: Work with Analytics, Data Science/ML, Risk, Fraud and Finance to turn business needs into reliable, documented data products.
  • Evaluation: Evaluate new Databricks capabilities and tools and recommend adoption where there is clear value.
  •  

REQUIRED SKILLS:

 

  • Experience: 12+ years in data or software engineering, with at least 5 years in a technical lead, staff or principal role.
  • SQL: Expert-level SQL: complex analytical queries, window functions, CTEs, query plan analysis and tuning on very large datasets (billions of rows).
  • Spark and big data: Deep experience with Apache Spark (PySpark and/or Scala) and the ecosystem: Hadoop, Hive, Kafka, Parquet/ORC/Delta.
  • Databricks in production: Hands-on production use of Delta Lake, Unity Catalog, Workflows/Jobs, Databricks SQL, and cluster and cost management.
  • Pipelines: Proven design and operation of large-scale batch and streaming pipelines, including Structured Streaming, Delta Live Tables and Kafka, on a major cloud.
  • Migration: Experience migrating on-premises or EMR-based Hadoop/Hive platforms to Databricks.
  • Performance and cost: Proven Spark and Databricks performance tuning and cost optimization, including cluster policies.
  • Data modeling: Dimensional, entity-centric, Data Vault or medallion modeling, including slowly changing dimensions, incremental/CDC loads and soft deletes.
  • Orchestration and CI/CD: Airflow, Databricks Workflows or Oozie; Git, Terraform or Databricks Asset Bundles, and automated testing and data quality checks.
  • Data security: Implementation of PII protection: encryption, tokenization, and row/column-level security.
  • Leadership and communication: Mentoring engineers, and explaining technical trade-offs clearly to engineers and executives.
  •  

EDUCATION: Graduate in Computer Science, Data Science, or a related field. 12+ years of experience in data modelling or data analysis.


KEY SKILLS:Databricks, Big Data Technologies

Role: Databricks Principal Engineer

Experience: 12+

Location: All EXL Location  

Work Mode: Hybrid 

POSITION SUMMARY


We are building our next-generation data platform on Databricks. The Principal Engineer sets the technical direction for how the bank ingests, models, governs and serves data. This is a hands-on role: you will design the detailed lakehouse implementation, set engineering standards for SQL and pipelines, lead the migration from legacy platforms, and mentor engineers across several teams. You will work within the target-state architecture owned by the Solution Architect.


ROLES AND RESPONSIBILITIES: 


  • Lakehouse design: Own the detailed design of the Databricks lakehouse: ingestion, bronze/silver/gold layers, Delta Lake design, Unity Catalog setup, and workspace and cluster configuration.
  • Migration: Lead the hands-on migration of legacy Hadoop/Hive/Spark and warehouse workloads to Databricks, including re-platforming HQL and SQL, validating data parity, and safe cutover.
  • Engineering standards: Set standards for SQL, PySpark and Delta Live Tables/Lakeflow pipelines, covering code review, testing, CI/CD and observability.
  • Performance and cost: Tune performance and cost: partitioning, liquid clustering/Z-ordering, file compaction, join and shuffle tuning, Photon, cluster right-sizing and DBU spend governance.
  • Data modeling: Design canonical, auditable entity models (applications, accounts, customers, transactions) across source systems.
  • Security and compliance: Implement PII protection, encryption and tokenization, row/column-level security, lineage, retention and audit readiness with Security, Risk, Compliance and Data Governance.
  • Design reviews: Lead technical design reviews, write architecture decision records, and resolve cross-team technical trade-offs.
  • Mentoring: Mentor senior and mid-level engineers and raise engineering quality across the data organization.
  • Data products: Work with Analytics, Data Science/ML, Risk, Fraud and Finance to turn business needs into reliable, documented data products.
  • Evaluation: Evaluate new Databricks capabilities and tools and recommend adoption where there is clear value.
  •  

REQUIRED SKILLS:

 

  • Experience: 12+ years in data or software engineering, with at least 5 years in a technical lead, staff or principal role.
  • SQL: Expert-level SQL: complex analytical queries, window functions, CTEs, query plan analysis and tuning on very large datasets (billions of rows).
  • Spark and big data: Deep experience with Apache Spark (PySpark and/or Scala) and the ecosystem: Hadoop, Hive, Kafka, Parquet/ORC/Delta.
  • Databricks in production: Hands-on production use of Delta Lake, Unity Catalog, Workflows/Jobs, Databricks SQL, and cluster and cost management.
  • Pipelines: Proven design and operation of large-scale batch and streaming pipelines, including Structured Streaming, Delta Live Tables and Kafka, on a major cloud.
  • Migration: Experience migrating on-premises or EMR-based Hadoop/Hive platforms to Databricks.
  • Performance and cost: Proven Spark and Databricks performance tuning and cost optimization, including cluster policies.
  • Data modeling: Dimensional, entity-centric, Data Vault or medallion modeling, including slowly changing dimensions, incremental/CDC loads and soft deletes.
  • Orchestration and CI/CD: Airflow, Databricks Workflows or Oozie; Git, Terraform or Databricks Asset Bundles, and automated testing and data quality checks.
  • Data security: Implementation of PII protection: encryption, tokenization, and row/column-level security.
  • Leadership and communication: Mentoring engineers, and explaining technical trade-offs clearly to engineers and executives.
  •  

EDUCATION: Graduate in Computer Science, Data Science, or a related field. 12+ years of experience in data modelling or data analysis.


KEY SKILLS:Databricks, Big Data Technologies


Responsibilities
  • Lakehouse design: Own the detailed design of the Databricks lakehouse: ingestion, bronze/silver/gold layers, Delta Lake design, Unity Catalog setup, and workspace and cluster configuration.
  • Migration: Lead the hands-on migration of legacy Hadoop/Hive/Spark and warehouse workloads to Databricks, including re-platforming HQL and SQL, validating data parity, and safe cutover.
  • Engineering standards: Set standards for SQL, PySpark and Delta Live Tables/Lakeflow pipelines, covering code review, testing, CI/CD and observability.
  • Performance and cost: Tune performance and cost: partitioning, liquid clustering/Z-ordering, file compaction, join and shuffle tuning, Photon, cluster right-sizing and DBU spend governance.
  • Data modeling: Design canonical, auditable entity models (applications, accounts, customers, transactions) across source systems.
  • Security and compliance: Implement PII protection, encryption and tokenization, row/column-level security, lineage, retention and audit readiness with Security, Risk, Compliance and Data Governance.
  • Design reviews: Lead technical design reviews, write architecture decision records, and resolve cross-team technical trade-offs.
  • Mentoring: Mentor senior and mid-level engineers and raise engineering quality across the data organization.
  • Data products: Work with Analytics, Data Science/ML, Risk, Fraud and Finance to turn business needs into reliable, documented data products.
  • Evaluation: Evaluate new Databricks capabilities and tools and recommend adoption where there is clear value.
Qualifications

Graduate in Computer Science, Data Science, or related field. 12+ years of experience in data engineering or related field.

Skills Required

  • Graduate degree in Computer Science, Data Science, or a related field
  • 12+ years of experience in data or software engineering
  • At least 5 years in a technical lead, staff, or principal role
  • Expert-level SQL, including analytical queries, window functions, CTEs, query-plan analysis, and tuning at large scale
  • Deep Apache Spark experience with PySpark and/or Scala
  • Experience with Hadoop, Hive, Kafka, Parquet, ORC, and Delta
  • Production experience with Databricks, Delta Lake, Unity Catalog, Workflows or Jobs, Databricks SQL, and cluster management
  • Experience designing and operating large-scale batch and streaming pipelines using Structured Streaming, Delta Live Tables, and Kafka
  • Experience migrating on-premises or EMR-based Hadoop and Hive platforms to Databricks
  • Experience with Spark and Databricks performance tuning and cost optimization
  • Experience with dimensional, entity-centric, Data Vault, or medallion data modeling
  • Experience with slowly changing dimensions, incremental or CDC loads, and soft deletes
  • Experience with Airflow, Databricks Workflows, or Oozie
  • Experience with Git, Terraform, or Databricks Asset Bundles
  • Experience implementing automated testing and data quality checks
  • Experience implementing PII protection, encryption, tokenization, and row- or column-level security
  • Ability to mentor engineers and communicate technical trade-offs to engineers and executives

EXL Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about EXL and has not been reviewed or approved by EXL.

  • Healthcare Strength — Medical plan options include HSA, HRA, and PPO choices, with additional health-related programs such as disability coverage and an employee assistance program. Feedback suggests these options provide meaningful flexibility and coverage breadth for many roles.
  • Leave & Time Off Breadth — Up to 20 days of paid time off, plus federal holidays and paid sick and safe time, are highlighted. Feedback suggests this leave structure is a notable benefit alongside core compensation.
  • Retirement Support — A traditional and Roth 401(k) with up to a 3% company match and full vesting after one year is described. Feedback suggests this match and vesting timeline help bolster long-term savings.

EXL Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
30,246 Employees
Year Founded: 1999

What We Do

Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.

Similar Jobs

Optum Logo Optum

Technical Product Manager

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Hyderabad, Telangana, IND
160000 Employees

Micron Technology Logo Micron Technology

Senior Manager, ITP Program Management, AI and Agentic Platform

Artificial Intelligence • Hardware • Information Technology • Machine Learning
In-Office
2 Locations
45000 Employees

Tapestry - Coach and Kate Spade Logo Tapestry - Coach and Kate Spade

Manager, Production Operations

eCommerce • Fashion • Retail • Sales • Wearables • Design
Remote or Hybrid
India
16000 Employees

Capco Logo Capco

Product Manager

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
India
6000 Employees

Similar Companies Hiring

NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity • Defense • Defense Technology
Austin, TX
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account