Lead Data Engineer - PySpark/SQL/AI

Posted Yesterday
Be an Early Applicant
Hyderabad, Telangana, IND
Hybrid
Senior level
Fintech • Financial Services
Wells Fargo: Tech-powered. Innovation-led. We're transforming financial services.
The Role
Leads enterprise data-platform modernization, migrating legacy warehouses, Hadoop, and ETL systems to scalable cloud-native lakehouse ecosystems. Designs Medallion architectures and batch, streaming, and real-time pipelines using Python, PySpark, and SQL. Establishes standards for reliability, observability, security, governance, data quality, and cost optimization. Drives migration planning, automation, reconciliation, testing, and cutover while enabling data products, Data Mesh, self-service analytics, AI capabilities, and collaboration across stakeholders and engineering teams.
Summary Generated by Built In
About this role:
Wells Fargo is seeking a Lead Data Engineer
In this role, you will:
  • Lead complex initiatives with broad impact and act as key participant in large scale software planning for the Technology area
  • Design, develop, and run tooling to discover problems in data and applications and report the issues to engineering and product leadership
  • Review and analyze complex software enhancement initiatives for business, operational or technical improvements that require in depth evaluation of multiple factors including intangibles or unprecedented factors
  • Make decisions in complex and multi-faceted data engineering situations requiring understanding of software package options and programming language and compliance requirements that influence and lead Technology to meet deliverables and drive organizational change
  • Strategically collaborate and consult with internal partners to resolve highly risky data engineering challenges
Required Qualifications:
  • 5+ years of Database Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
Desired Qualifications:
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field.
  • 5 - 10+ years of experience in Data Engineering and Enterprise Data Platforms.
  • 5+ years leading enterprise-scale data modernization initiatives.
  • Strong hands-on expertise with:
    • Python
    • Apache Spark / PySpark
    • SQL
    • ETL/ELT Design
    • API Integration
  • Deep understanding of:
    • Lakehouse Architecture
    • Medallion Architecture
    • Data Warehousing Concepts
    • Data Governance
    • Metadata Management
    • Data Quality Frameworks
  • Proven experience with any cloud platforms:
    • Azure
    • Microsoft Fabric
    • Databricks
    • GCP
    • Experience migrating large-scale data environments from on-premise to cloud.
  • Experience in HR, Workforce, Talent, Payroll, or Enterprise Data domains.
  • Exposure to AI/ML, Generative AI, Agentic AI, and LLM-based application development.
  • Experience implementing Data Products, Data Mesh, and Self-Service Analytics platforms.
  • Knowledge of Kubernetes, Docker, Terraform, and Infrastructure as Code.
  • Familiarity with MLOps, DataOps, and DevSecOps practices.
  • Cloud certifications in Azure, Databricks, GCP, or equivalent.
Job Expectations:
We are seeking a highly experienced Modern Data Engineering Lead to drive the modernization of HR Data platforms and accelerate the adoption of Cloud, AI, and Data Engineering best practices. This role will lead the transformation of large-scale legacy data ecosystems into scalable, cloud-native, AI-enabled platforms while enabling advanced analytics, data products, automation, and self-service capabilities.
The ideal candidate will possess deep expertise in Cloud Data Platforms, Lakehouse Architecture, Data Migration, AI-powered Engineering, Python, Spark, API integration, and Enterprise Data Modernization. This leader will work closely with business stakeholders, architects, product teams, and engineering organizations to deliver strategic outcomes across HR Data systems.
Data Platform Modernization
  • Lead enterprise-scale modernization initiatives from legacy data warehouses and ETL platforms to cloud-native data ecosystems.
  • Design and implement modern Lakehouse architectures using Medallion (Bronze, Silver, Gold) data design principles.
  • Develop scalable and secure data platforms supporting batch, streaming, and real-time processing workloads.
  • Drive architecture decisions involving Data Lake, Data Warehouse, Lakehouse, Data Mesh, and Data Fabric patterns.
Cloud & Data Engineering
  • Design, develop, and optimize large-scale data pipelines using:
    • Python
    • Apache Spark / PySpark
    • SQL
    • ETL/ELT frameworks
  • Implement cloud-native solutions leveraging platforms such as:
    • Microsoft Fabric
    • Establish engineering standards for reliability, scalability, observability, and cost optimization.
Data Migration & Transformation
  • Lead large-scale migration programs involving:
    • Legacy Data Warehouses
    • On-Prem Hadoop/Cloudera Platforms
    • Traditional ETL Middleware
  • Develop migration strategies including assessment, planning, automation, reconciliation, testing, and cutover execution.
  • Ensure business continuity and data integrity during migration programs.
Posting End Date:
29 Aug 2026
*Job posting may come down early due to volume of applicants.
We Value Equal Opportunity
Wells Fargo is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other legally protected characteristic.
Employees support our focus on building strong customer relationships balanced with a strong risk mitigating and compliance-driven culture which firmly establishes those disciplines as critical to the success of our customers and company. They are accountable for execution of all applicable risk programs (Credit, Market, Financial Crimes, Operational, Regulatory Compliance), which includes effectively following and adhering to applicable Wells Fargo policies and procedures, appropriately fulfilling risk and compliance obligations, timely and effective escalation and remediation of issues, and making sound risk decisions. There is emphasis on proactive monitoring, governance, risk identification and escalation, as well as making sound risk decisions commensurate with the business unit's risk appetite and all risk and compliance program requirements.
Candidates applying to job openings posted in Canada: Applications for employment are encouraged from all qualified candidates, including women, persons with disabilities, aboriginal peoples and visible minorities. Accommodation for applicants with disabilities is available upon request in connection with the recruitment process.
Applicants with Disabilities
To request a medical accommodation during the application or interview process, visit Disability Inclusion at Wells Fargo .
Drug and Alcohol Policy
Wells Fargo maintains a drug free workplace. Please see our Drug and Alcohol Policy to learn more.
Wells Fargo Recruitment and Hiring Requirements:
a. Third-Party recordings are prohibited unless authorized by Wells Fargo.
b. Wells Fargo requires you to directly represent your own experiences during the recruiting and hiring process.

Skills Required

  • 5+ years of database engineering experience, or equivalent experience, training, military experience, or education
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field
  • 5–10+ years of experience in data engineering and enterprise data platforms
  • 5+ years leading enterprise-scale data modernization initiatives
  • Hands-on expertise with Python, Apache Spark/PySpark, SQL, ETL/ELT design, and API integration
  • Understanding of lakehouse, Medallion, data warehousing, data governance, metadata management, and data quality frameworks
  • Experience with Azure, Microsoft Fabric, Databricks, or GCP cloud platforms
  • Experience migrating large-scale data environments from on-premises to cloud
  • Experience in HR, workforce, talent, payroll, or enterprise data domains
  • Exposure to AI/ML, Generative AI, Agentic AI, and LLM-based application development
  • Experience implementing data products, Data Mesh, and self-service analytics platforms
  • Knowledge of Kubernetes, Docker, Terraform, and Infrastructure as Code
  • Familiarity with MLOps, DataOps, and DevSecOps practices
  • Cloud certification in Azure, Databricks, GCP, or equivalent
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
205,000 Employees
Year Founded: 1852

What We Do

Wells Fargo & Company (NYSE: WFC) is a leading financial services company that has approximately $2.2 trillion in assets. We provide a diversified set of banking, investment and mortgage products and services, as well as consumer and commercial finance, through our four reportable operating segments: Consumer Banking and Lending, Commercial Banking, Corporate and Investment Banking, and Wealth & Investment Management. Wells Fargo ranked No. 33 on Fortune’s 2025 rankings of America’s largest corporations. Our technology professionals drive innovation, information security, and big data analytics while maintaining a network that handles more than 12 billion customer interactions a year. Join us! Are you looking for more? Find it here. At Wells Fargo, we're more than a financial services leader – we’re a global trailblazer committed to driving innovation, empowering communities, and helping our customers succeed. We believe that a meaningful career is much more than just a job – it’s about finding all of the elements to help you thrive, in one place. Living the Well Life means you’re supported in life, not just work. It means having robust benefits, competitive compensation, and programs designed to help you find work-life balance and well-being. You’ll be rewarded for investing in your community, celebrated for being your authentic self, and empowered to grow. And we’re recognized for it — Wells Fargo continues to rank on the LinkedIn Top Companies lists of best workplaces “to grow your career.” All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other legally protected characteristic. © 2026 Wells Fargo Bank, N.A. All rights reserved. Member FDIC.

Why Work With Us

We're known for our “Well Life” approach to supporting employees’ career aspirations, work-life balance, and mental and physical health. Wells Fargo continues to rank on the LinkedIn Top Companies lists of best workplaces “to grow your career.”

Gallery

Gallery
Gallery
Gallery
Gallery

Wells Fargo Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Typical time on-site: 3 days a week
HQSan Francisco, CA
Bangalore, Bangalore
Belfast, GB
Bengaluru, Karnataka
Chandler, AZ
Charlotte, NC
Technology Center
Hyderabad, Telangana
Irving, TX
New York, NY
New York, NY
Phoenix, AZ
Learn more

Similar Jobs

Wells Fargo Logo Wells Fargo

Senior Software Engineer

Fintech • Financial Services
Hybrid
Hyderabad, Telangana, IND
205000 Employees
Hybrid
Hyderabad, Telangana, IND
205000 Employees

Wells Fargo Logo Wells Fargo

Executive Assistant

Fintech • Financial Services
Hybrid
Hyderabad, Telangana, IND
205000 Employees

Wells Fargo Logo Wells Fargo

Senior Product Manager

Fintech • Financial Services
Hybrid
Hyderabad, Telangana, IND
205000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account