Data Engineering Lead

Posted 5 Hours Ago
Be an Early Applicant
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
In-Office
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
The Role
Leads enterprise data engineering and analytics initiatives using Databricks, Delta Lake, Spark, and PySpark. Designs dimensional, semantic, and context-layer models; builds scalable ETL/ELT pipelines; optimizes performance and cloud costs; and establishes data governance, quality, lineage, and monitoring standards. Partners with technical and business stakeholders, mentors engineers, conducts architecture reviews, and influences enterprise data platform strategy. Supports reporting, self-service analytics, AI/ML, and regulated healthcare data use cases.
Summary Generated by Built In
Requisition Number: 2387751
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.
Primary Responsibilities:
  • Data Modeling & Analytics Engineering
    • Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
    • Build curated data products that support reporting, analytics, AI, and machine learning use cases
    • Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
    • Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
    • Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
  • Databricks Platform Engineering
    • Design and implement modern data architectures leveraging Databricks Lakehouse principles
    • Develop and maintain Bronze, Silver, and Gold data layers
    • Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
    • Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)
  • Delta optimization techniques
    • Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
    • Context Layer & Semantic Layer Development
    • Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
    • Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
    • Enable self-service analytics through Power BI, Tableau, and AI-driven applications
    • Manage semantic models and governance processes to ensure reporting consistency
    • Establish metric certification, lineage tracking, and data governance standards
  • Performance Optimization & Cost Management
    • Analyze query execution plans and identify performance bottlenecks
    • Optimize Spark workloads for scalability, reliability, and cost efficiency
    • Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies
    • Tune Delta Lake tables through file compaction, partition optimization, and storage management
    • Monitor warehouse and cluster utilization to improve performance and control cloud expenses
    • Enhance dashboard and reporting performance through optimized data models and query design
  • Data Governance & Quality Management
    • Develop and implement data quality frameworks, validation processes, and automated monitoring
    • Ensure compliance with enterprise security, governance, and regulatory requirements
    • Support metadata management, data cataloging, and lineage initiatives
    • Establish monitoring and alerting for data pipelines, processing failures, and data anomalies
    • Partner with governance teams to drive data stewardship and certification processes
  • Technical Leadership & Collaboration
    • Collaborate with data scientists, analysts, engineers, product owners, and business leaders
    • Mentor and guide junior analytics engineers and data engineers
    • Conduct architecture reviews and recommend technology best practices
    • Drive adoption of enterprise data standards, frameworks, and governance processes
    • Lead technical design discussions and influence data platform strategy and roadmap
  • Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:
  • Graduate degree or equivalent experience
  • Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field
  • 7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines
  • 3+ years of hands-on experience with Databricks and Delta Lake technologies
  • Hands-on experience with Power BI, Tableau, or similar BI platforms
  • Experience building enterprise-scale ETL/ELT pipelines
  • Experience implementing semantic layers and enterprise reporting solutions
  • Experience with performance tuning of Spark workloads and large-scale analytics environments
  • Experience with cloud platforms such as Azure, AWS, or Google Cloud
  • Knowledge of CI/CD, DevOps practices, and version control systems such as Git
  • Solid understanding of Databricks Lakehouse architecture and Medallion (Bronze/Silver/Gold) design patterns
  • Solid understanding of data governance, metadata management, and data quality frameworks
  • Solid expertise in dimensional data modeling, including Star and Snowflake schema design
  • Advanced proficiency in SQL and Spark/PySpark development
  • Proven excellent problem-solving, communication, and stakeholder management skills

Preferred Qualifications:
  • Master's degree in Computer Science, Data Science, Analytics, or a related discipline
  • Databricks certifications (Data Engineer Associate, Professional, or equivalent)
  • Experience with Databricks Unity Catalog and enterprise data governance implementations
  • Experience supporting AI/ML, Generative AI, or advanced analytics use cases
  • Experience designing enterprise semantic models and metric layers
  • Healthcare, insurance, or highly regulated industry experience
  • Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy
  • Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation

Key Competencies:
  • Data Architecture & Modeling
  • Analytics Engineering
  • Databricks & Delta Lake
  • Spark/PySpark Development
  • Data Governance & Quality
  • Performance Optimization
  • Semantic Layer Design
  • Cloud Data Platforms
  • Technical Leadership
  • Stakeholder Management

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.

Skills Required

  • Graduate degree or equivalent experience
  • Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field
  • 7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines
  • 3+ years of hands-on experience with Databricks and Delta Lake technologies
  • Hands-on experience with Power BI, Tableau, or similar BI platforms
  • Experience building enterprise-scale ETL/ELT pipelines
  • Experience implementing semantic layers and enterprise reporting solutions
  • Experience with performance tuning of Spark workloads and large-scale analytics environments
  • Experience with cloud platforms such as Azure, AWS, or Google Cloud
  • Knowledge of CI/CD, DevOps practices, and version control systems such as Git
  • Solid understanding of Databricks Lakehouse architecture and Medallion Bronze/Silver/Gold design patterns
  • Solid understanding of data governance, metadata management, and data quality frameworks
  • Solid expertise in dimensional data modeling, including Star and Snowflake schema design
  • Advanced proficiency in SQL and Spark/PySpark development
  • Excellent problem-solving, communication, and stakeholder management skills
  • Master's degree in Computer Science, Data Science, Analytics, or a related discipline
  • Databricks certifications such as Data Engineer Associate or Professional
  • Experience with Databricks Unity Catalog and enterprise data governance implementations
  • Experience supporting AI/ML, Generative AI, or advanced analytics use cases
  • Experience designing enterprise semantic models and metric layers
  • Healthcare, insurance, or highly regulated industry experience
  • Experience leading technical teams, mentoring engineers, and driving enterprise data strategy
  • Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation

What the Team is Saying

Optum Compensation & Benefits Highlights

  • Leave & Time Off Breadth — PTO is generally described as decent or good, and many note it as a strong part of the package. Actual ability to take time off can depend on workload and team coverage.
  • Retirement Support — Offerings include a 401(k) with employer match and access to an employee stock purchase plan, which are highlighted as meaningful components of total rewards. These programs are consistently referenced among core benefits.
  • Parental & Family Support — Parental and caregiver leave, along with adoption assistance, are publicly highlighted and viewed as notable elements of the package. Availability can be role-specific, but these supports contribute to overall breadth where offered.

Optum Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Eden Prairie, MN
160,000 Employees
Year Founded: 2011

What We Do

Optum, part of the UnitedHealth Group family of businesses, is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. At Optum, we support your well-being with an understanding team, extensive benefits and rewarding opportunities. By joining us, you’ll have the resources to drive system transformation while we help you take care of your future. We recognize the power of connection to drive change, improve efficiency and make a difference in health care. Join a team where your skills and ideas can make an impact and where collaboration is key to creating technology that produces healthier outcomes.

Gallery

Gallery
Gallery
Gallery

Optum Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Optum has three workplace models that balance the needs of the business and the responsibilities of each role. These models, core on‑site (5 days/week), hybrid (4 days/week) and telecommute or fully remote, vary by country, role and location.

Typical time on-site: Not Specified
HQEden Prairie, MN
Metro Manila, Philippines
Cebu, Philippines
Davao, Philippines
Ann Arbor, MI
Atlanta, GA
Baltimore, MD
Bengaluru, India
Chennai, India
Dallas, TX
Detroit, MI
Dublin, Ireland
Hartford, CT
Houston, TX
Hyderabad, India
Jacksonville, FL
Las Vegas, NV
Letterkenny, Ireland
Louisville, KY
Madison, WI
Minneapolis, MN
Nashville, TN
New Delhi, India
Philadelphia, PA
Phoenix, AZ
Pune, India
Raleigh, NC
San Diego, CA
Washington, DC
Learn more

Similar Jobs

Optum Logo Optum

Software Engineering Lead - Postgre SQL

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Instructional Designer

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Principal Data Scientist

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Engineering Manager

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account