Top Senior Data Engineer Jobs

Designs and develops complex cloud-native data solutions, scalable batch and real-time pipelines, APIs, data models, and enterprise data platforms. Implements data governance, quality, lineage, security, observability, CI/CD, automated testing, and operational reporting capabilities. Collaborates with architects, product owners, analysts, governance, security, and platform teams on technology strategy and cloud modernization. Mentors engineers, leads technical design discussions, presents recommendations, and supports AI-ready data products and emerging development tools.

Builds scalable data pipelines, ETL workflows, data models, governance frameworks, and cloud migration solutions. Develops and tests code using Scala, Spark, PySpark, Hadoop, BigQuery, DataStage, Teradata, Kafka, and related technologies. Supports data quality, analytics, machine learning classification, workflow orchestration, system integration, troubleshooting, documentation, and stakeholder recommendations. The role also mentors less experienced programmers and contributes to production deployments.

Designs and maintains scalable data pipelines, ETL workflows, analytical data models, and cloud data solutions. Builds reporting-ready datasets for business intelligence and self-service analytics using Python, SQL, Spark, Hive, and cloud platforms. Optimizes queries and pipelines, validates data quality, performs data exploration, supports Tableau and Looker, documents data architecture, and partners with stakeholders to deliver reliable analytical insights.

Designs and develops large-scale data structures, ETL pipelines, databases, and data infrastructure for analytics and reporting. Deploys machine learning and statistical solutions, integrates models into production systems, and improves data reliability and quality. Acts as a data engineering subject matter expert, collaborates with executive, product, design, and data science teams, supports technical initiatives, and mentors junior engineers.

Manage the health, performance, reliability, and cost of customer Snowflake and Databricks environments. Responsibilities include customer onboarding, account and warehouse setup, permissions, data ingestion, query optimization, cost reporting, incident response, escalation support, and automation using Python and Terraform. The role also advises customers and internal teams on data platform best practices and participates in on-call rotations.

Design, develop, deploy, maintain, and optimize data solutions, including ETL/ELT pipelines, stored procedures, complex SQL, data models, and single- and multi-tenant platforms. Diagnose and refactor legacy processes, perform QA and testing, support CI/CD, collaborate across teams, and deliver scalable data solutions using Snowflake, MySQL, Python, and cloud technologies.

Lead the technical direction and production operations of Handshake’s data platform, including Airflow orchestration, ingestion, batch and streaming pipelines, BigQuery, Kubernetes, and cloud infrastructure. Build reliable workflow development and deployment paths, improve data resilience and observability, establish data contracts, support AI workloads, troubleshoot incidents, and mentor engineers across application, analytics, ML, and cloud teams.

Designs, builds, and operates large-scale Hadoop-based data pipelines and analytics platforms. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark applications; managing streaming, relational, and NoSQL data systems; orchestrating workflows; and ensuring reliable, performant production data platforms. The role also involves troubleshooting, documentation, and potentially operating cloud, lakehouse, Kubernetes, CI/CD, and infrastructure-as-code environments.

Design, build, and support a near-real-time transactional cache platform. Develop CDC pipelines from UniVerse to PostgreSQL, transform data for MongoDB Atlas, optimize data models and queries, and ensure platform reliability, freshness, and scalability. Partner with API and application teams, troubleshoot production issues, maintain monitoring and runbooks, support incident response, and mentor junior engineers. The role requires cloud-native data engineering across GCP, operational data stores, distributed systems, and production ETL/ELT pipelines.

Leads the design, implementation, and support of a near-real-time transactional cache platform. Builds CDC pipelines from UniVerse to PostgreSQL, transforms data into optimized MongoDB models, and supports API-serving workloads on GCP. Owns performance, reliability, observability, incident response, monitoring, and operational support. Provides technical leadership, establishes engineering standards, drives modernization, collaborates with application teams, and mentors engineers across data platform initiatives.

Designs, develops, and supports scalable ETL pipelines and large-scale data platforms using Python, PySpark, SQL Server, SSIS, and Hadoop technologies. Responsibilities include database performance tuning, data modeling, production troubleshooting, batch scheduling, data-quality controls, CI/CD, secure development, and collaboration across technical teams. The role also involves mentoring developers and evaluating AI-assisted development tools.

Leads the design, development, optimization, and security of very large SQL Server databases and data warehouses. Oversees ETL and Microsoft BI pipeline development, data modeling, performance tuning, code reviews, and access management. Collaborates with DBAs, mentors developers, applies AI-assisted optimization, assesses risks, and drives scalable, reliable data products.

Leads development and maintenance of telematics data systems supporting predictive and condition-based maintenance. Builds ETL pipelines, databases, integrations, semantic models, and Power BI reports; analyzes asset health data to identify failure patterns and improve reliability. Maintains data quality and workflow performance, develops predictive maintenance thresholds, performs root cause analysis, and collaborates with engineering, maintenance, reliability, and IT teams to optimize asset performance.

Designs, develops, tests, and documents scalable data architectures, applications, ETL pipelines, and data warehouses. Improves data delivery infrastructure, automates processes, troubleshoots data issues, and ensures data reliability, quality, and validation. Collaborates with developers, architects, analysts, data scientists, and business teams to deliver effective data solutions. The role also requires database performance tuning, maintenance, monitoring, and occasional work outside normal hours.

Build and maintain data lakes, data warehouses, and modern data pipelines for a multi-tenant SaaS platform. Design end-to-end data processes, write pipeline and infrastructure code, conduct design reviews, optimize performance, monitor systems, support data governance, and troubleshoot scalability and security issues. Collaborate with technical and business stakeholders, mentor junior engineers, evaluate technologies, document platform architecture, automate testing, and participate in on-call rotation.

Design and build a company-wide data platform supporting ingestion, CDC, replication, transformation, governance, semantic discovery, analytics, and self-service dbt workflows. Develop scalable architectures across distributed SQL databases and analytical platforms, establish data quality and observability standards, implement access controls and metadata foundations, and provide technical leadership across engineering teams. The role also shapes source-of-truth architecture and enables data for AI and agentic applications.

Designs and maintains scalable data infrastructure, ETL/ELT pipelines, reporting platforms, and dashboards. The role uses SQL, Python, Tableau, cloud platforms, and distributed data technologies to deliver self-service analytics, validate data quality, optimize pipeline performance, and support reliable data systems. Responsibilities include stakeholder collaboration, requirements gathering, KPI visualization, technical documentation, operational monitoring, and mentoring junior engineers and analysts.

Develop and monitor scalable analytics frameworks for Pega Customer Decision Hub using Databricks, Python, PySpark, and SQL. Build reusable notebooks, pipelines, queries, and monitoring solutions for customer interaction analytics, model performance, attribution, propensity analysis, and engagement effectiveness. Standardize analytical processes, support back-testing and KPI development, and enable self-service analytics across marketing and enterprise teams.

Designs, builds, and operates large-scale Hadoop data pipelines and analytics platforms. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark-based processing solutions; managing streaming workflows; working with relational and NoSQL databases; orchestrating jobs; troubleshooting distributed systems; and supporting reliable, scalable production data platforms. Preferred experience includes cloud Hadoop environments, lakehouse formats, governance tools, Kubernetes, CI/CD, and infrastructure as code.

Build and own trusted data products, dimensional models, semantic layers, and dbt transformations that support analytics, experimentation, machine learning, reporting, and decision-making. Partner with cross-functional teams to translate business needs into scalable datasets, improve data quality and governance, establish engineering standards, and drive self-service analytics. Mentor engineers and analysts while leveraging AI tools to improve development, documentation, and quality.

Design, build, and operate WHOOP’s data platform infrastructure, including lakehouse, warehouse, and streaming systems. Improve performance, reliability, cost efficiency, observability, and developer experience across AWS-based data systems. Develop internal tools, APIs, automation, CI/CD, and infrastructure-as-code practices. Partner with data, ML, analytics, and software engineering teams to support scalable workflows, governance, and production excellence while using AI-assisted development tools effectively.

Own the architecture, roadmap, and operation of Clarium’s AWS-based data platform. Build reliable batch and near-real-time pipelines, model healthcare supply chain data, optimize Postgres and Snowflake performance, and establish standards for testing, deployment, quality, and governance. Partner cross-functionally to create durable data models, lead multi-quarter initiatives, manage sensitive data controls, and mentor engineers.

Leads the architecture, development, operation, validation, and optimization of high-volume ETL and data migration pipelines. Transforms structured and unstructured data from legacy systems into cloud platforms, troubleshoots failures, performs data quality validation, conducts root cause analysis, and documents migration processes. Collaborates with cloud, DevSecOps, database, QA, and product teams while reporting pipeline performance, migration status, and data quality metrics.

Designs, builds, and optimizes scalable data pipelines, warehouses, lakes, and platforms integrating complex network operations data. Uses Kafka, Elastic Stack, AWS, Kubernetes, and automation to support ingestion, transformation, analytics, dashboards, security, and compliance. Responsibilities include ETL/ELT, data validation, CI/CD, performance tuning, troubleshooting, documentation, and collaboration with analysts and visualization engineers. The role requires an active TS/SCI clearance, Security+ certification, and extensive data engineering experience in DoD or similarly secure environments.

Designs, builds, and optimizes scalable data pipelines, warehouses, lakes, and ingestion platforms using Kafka, Elastic Stack, AWS, and Kubernetes. Integrates disparate network operations data while ensuring quality, security, and DoD compliance. Automates ETL/ELT, validation, monitoring, CI/CD, and deployment processes, supports dashboards and analytics, troubleshoots platform performance, and documents architecture and operational procedures.