Lead Data Engineer

Posted Yesterday
Be an Early Applicant
Hiring Remotely in United States
Remote or Hybrid
Mid level
Information Technology • Database • Consulting
The Role
Lead data engineering teams to design, build, and maintain scalable cloud-native data pipelines, data platforms, and data products using Snowflake, Databricks, Airflow, and modern architectures. Optimize ETL/ELT, SQL, and transformations for analytics, reporting, and AI use cases; ensure data quality, governance, observability, and stakeholder alignment. Produce documentation, runbooks, and present technical solutions to business and technical audiences.
Summary Generated by Built In

As a technical lead, you will guide engineering teams, establish best practices, and ensure successful delivery across multiple workstreams while enabling data-driven decision-making across the enterprise.
 

Responsibilities

Data Engineering & Platform Development

  • Design, develop, and maintain scalable data pipelines and data products supporting analytics, reporting, AI, and operational use cases.
  • Build and optimize ETL/ELT frameworks for large-scale data ingestion, transformation, validation, and consumption.
  • Develop and manage cloud-native data platforms leveraging Snowflake, AWS, Apache Airflow and modern data architectures.
  • Create scalable data models, data marts, semantic layers, and curated datasets that support enterprise analytics initiatives.
  • Optimize SQL workloads, transformation logic, and query performance to improve scalability and cost efficiency.
  • Establish reusable engineering frameworks, accelerators, and best practices to improve delivery consistency across projects.
  • Ensure high standards of data quality, reliability, governance, and observability throughout the data lifecycle.
  • Develop and maintain Snowflake-based data ecosystems, leveraging advanced features for performance optimization and data sharing.
  • Build and orchestrate data workflows using Airflow and other workflow scheduling platforms.
  • Collaborate directly with client stakeholders to gather requirements, define roadmaps, and develop scalable technical solutions.
  • Present solution designs, technical recommendations, and project updates to both technical and business audiences.
  • Prepare and maintain comprehensive project documentation, technical specifications, architecture diagrams, and operational runbooks.
Qualifications

Required Qualifications

  • 4+ years of experience in Data Engineering, Big Data Engineering, or Cloud Data Platform development.
  • Bachelor's or Master's degree in Computer Science, Engineering, Analytics, Mathematics, Information Systems, or related disciplines.
  • Strong hands-on expertise in SQL, Python, and PySpark.
  • Extensive experience working with Snowflake, Databricks, or similar cloud-native data platforms.
  • Proven experience building and supporting large-scale ETL/ELT data pipelines.
  • Strong understanding of data warehousing concepts, dimensional modeling, and modern Lakehouse architectures.
  • Experience implementing Medallion Architecture and enterprise-grade data modeling practices.
  • Hands-on experience with workflow orchestration tools such as Apache Airflow or equivalent scheduling frameworks.
  • Experience working with cloud ecosystems including AWS, Azure, or GCP.
  • Strong knowledge of performance tuning, optimization, monitoring, and operational support for data platforms.
  • Demonstrated experience leading engineering teams and coordinating with client and internal stakeholders.
  • Excellent analytical, problem-solving, communication, and stakeholder management skills.
  • Ability to work independently and lead complex initiatives in fast-paced consulting environments.

    Preferred Qualifications

  • Experience with streaming and real-time data processing frameworks.
  • Familiarity with DataOps, CI/CD, Infrastructure as Code, and DevOps practices.
  • Experience with data governance, data quality frameworks, and metadata management.
  • Exposure to AI/ML data pipelines and feature engineering workflows.
  • Experience with visualization tools such as Tableau, Power BI, or Looker.
  • Hands-on experience with Big Data technologies including Spark, Hadoop, Hive, HBase, Kafka, or related platforms.
  • Consulting or client-facing delivery experience in enterprise-scale environments.



Skills Required

  • 4+ years of experience in Data Engineering, Big Data Engineering, or Cloud Data Platform development.
  • Bachelor's or Master's degree in Computer Science, Engineering, Analytics, Mathematics, Information Systems, or related disciplines.
  • Strong hands-on expertise in SQL, Python, and PySpark.
  • Extensive experience working with Snowflake, Databricks, or similar cloud-native data platforms.
  • Proven experience building and supporting large-scale ETL/ELT data pipelines.
  • Strong understanding of data warehousing concepts, dimensional modeling, and modern Lakehouse architectures.
  • Experience implementing Medallion Architecture and enterprise-grade data modeling practices.
  • Hands-on experience with workflow orchestration tools such as Apache Airflow or equivalent scheduling frameworks.
  • Experience working with cloud ecosystems including AWS, Azure, or GCP.
  • Strong knowledge of performance tuning, optimization, monitoring, and operational support for data platforms.
  • Demonstrated experience leading engineering teams and coordinating with client and internal stakeholders.
  • Excellent analytical, problem-solving, communication, and stakeholder management skills.
  • Ability to work independently and lead complex initiatives in fast-paced consulting environments.
  • Experience with streaming and real-time data processing frameworks.
  • Familiarity with DataOps, CI/CD, Infrastructure as Code, and DevOps practices.
  • Experience with data governance, data quality frameworks, and metadata management.
  • Exposure to AI/ML data pipelines and feature engineering workflows.
  • Experience with visualization tools such as Tableau, Power BI, or Looker.
  • Hands-on experience with Big Data technologies including Spark, Hadoop, Hive, HBase, Kafka, or related platforms.
  • Consulting or client-facing delivery experience in enterprise-scale environments.
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
30,246 Employees
Year Founded: 1999

What We Do

Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.

Similar Jobs

In-Office or Remote
Deerfield, IL, USA
10000 Employees
161K-193K Annually

Trimble Logo Trimble

Lead Data Engineer

Hardware • Information Technology • Other • Software • Analytics
Remote
TX, USA
10001 Employees
136K-188K Annually

Stryker Logo Stryker

Lead Data Engineer

Healthtech • Other • Robotics • Biotech • Manufacturing
In-Office or Remote
4 Locations
51000 Employees
118K-256K Annually
In-Office or Remote
4 Locations
956 Employees
118K-256K Annually

Similar Companies Hiring

Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees
Standard Template Labs Thumbnail
Artificial Intelligence • Information Technology • Software
New York, NY
25 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account