Top Data Engineer Jobs

Lead the technical direction and production operations of Handshake’s data platform, including Airflow orchestration, ingestion, batch and streaming pipelines, BigQuery, Kubernetes, and cloud infrastructure. Build reliable workflow development and deployment paths, improve data resilience and observability, establish data contracts, support AI workloads, troubleshoot incidents, and mentor engineers across application, analytics, ML, and cloud teams.

Designs, develops, and supports scalable ETL pipelines and large-scale data platforms using Python, PySpark, SQL Server, SSIS, and Hadoop technologies. Responsibilities include database performance tuning, data modeling, production troubleshooting, batch scheduling, data-quality controls, CI/CD, secure development, and collaboration across technical teams. The role also involves mentoring developers and evaluating AI-assisted development tools.

Leads the design, development, optimization, and security of very large SQL Server databases and data warehouses. Oversees ETL and Microsoft BI pipeline development, data modeling, performance tuning, code reviews, and access management. Collaborates with DBAs, mentors developers, applies AI-assisted optimization, assesses risks, and drives scalable, reliable data products.

Build and maintain data lakes, data warehouses, and modern data pipelines for a multi-tenant SaaS platform. Design end-to-end data processes, write pipeline and infrastructure code, conduct design reviews, optimize performance, monitor systems, support data governance, and troubleshoot scalability and security issues. Collaborate with technical and business stakeholders, mentor junior engineers, evaluate technologies, document platform architecture, automate testing, and participate in on-call rotation.

Design and build a company-wide data platform supporting ingestion, CDC, replication, transformation, governance, semantic discovery, analytics, and self-service dbt workflows. Develop scalable architectures across distributed SQL databases and analytical platforms, establish data quality and observability standards, implement access controls and metadata foundations, and provide technical leadership across engineering teams. The role also shapes source-of-truth architecture and enables data for AI and agentic applications.

Design, build, and operate WHOOP’s data platform infrastructure, including lakehouse, warehouse, and streaming systems. Improve performance, reliability, cost efficiency, observability, and developer experience across AWS-based data systems. Develop internal tools, APIs, automation, CI/CD, and infrastructure-as-code practices. Partner with data, ML, analytics, and software engineering teams to support scalable workflows, governance, and production excellence while using AI-assisted development tools effectively.

Own the architecture, roadmap, and operation of Clarium’s AWS-based data platform. Build reliable batch and near-real-time pipelines, model healthcare supply chain data, optimize Postgres and Snowflake performance, and establish standards for testing, deployment, quality, and governance. Partner cross-functionally to create durable data models, lead multi-quarter initiatives, manage sensitive data controls, and mentor engineers.

Execute ETL workflows supporting large-scale migration from legacy systems to cloud environments. Extract, transform, cleanse, validate, and load structured and unstructured data; monitor pipelines; troubleshoot defects; and support automation, quality controls, staging, testing, and cutover operations. Maintain data maps, logs, procedures, and technical documentation while collaborating with data, cloud, QA, and product teams in a federal modernization program.

Leads the architecture, development, operation, validation, and optimization of high-volume ETL and data migration pipelines. Transforms structured and unstructured data from legacy systems into cloud platforms, troubleshoots failures, performs data quality validation, conducts root cause analysis, and documents migration processes. Collaborates with cloud, DevSecOps, database, QA, and product teams while reporting pipeline performance, migration status, and data quality metrics.

Designs, builds, and optimizes scalable data pipelines, warehouses, lakes, and platforms integrating complex network operations data. Uses Kafka, Elastic Stack, AWS, Kubernetes, and automation to support ingestion, transformation, analytics, dashboards, security, and compliance. Responsibilities include ETL/ELT, data validation, CI/CD, performance tuning, troubleshooting, documentation, and collaboration with analysts and visualization engineers. The role requires an active TS/SCI clearance, Security+ certification, and extensive data engineering experience in DoD or similarly secure environments.

Designs, builds, and optimizes scalable data pipelines, warehouses, lakes, and ingestion platforms using Kafka, Elastic Stack, AWS, and Kubernetes. Integrates disparate network operations data while ensuring quality, security, and DoD compliance. Automates ETL/ELT, validation, monitoring, CI/CD, and deployment processes, supports dashboards and analytics, troubleshoots platform performance, and documents architecture and operational procedures.

Designs, builds, and maintains scalable data pipelines, warehouses, lakes, and integrations using Kafka, Elastic Stack, AWS, and Kubernetes. Optimizes ingestion, querying, indexing, storage, and platform performance while supporting visualization, CI/CD, automation, security, and DoD/DISA compliance. Collaborates across engineering, analytics, software, and network operations teams, documents architectures and procedures, and resolves data-platform issues in classified enterprise environments.

Builds and maintains scalable data pipelines, warehouses, and lakes for analytics and reporting. Integrates network operations data using Kafka, Elastic Stack, AWS, and OpenShift/Kubernetes; develops ETL/ELT automation, dashboards, CI/CD processes, and data validation workflows. Optimizes ingestion, queries, indexing, storage, and cluster performance while ensuring security and DoD/DISA compliance. Troubleshoots platform issues, collaborates with analysts and visualization engineers, and documents architectures and operational procedures.

Leads and mentors the Data Engineering team while remaining hands-on in designing, building, and maintaining scalable data pipelines, warehouses, integrations, and infrastructure. Establishes architecture, reliability, quality, and performance standards; guides technical delivery; and resolves complex issues. Partners with Finance and business stakeholders to translate data needs into practical solutions, sets technical direction, evaluates technologies, and communicates strategy, risks, and recommendations to leadership.

Leads and delivers complex data engineering projects, designing scalable data infrastructure, pipelines, warehouses, and data quality solutions. Recommends tools and architectures, optimizes data movement and performance, collaborates across technical teams, mentors engineers, and translates technical concepts for clients and nontechnical stakeholders. The role may also involve real-time processing, data modeling, transformation, security, and MLOps.

Design, develop, test, and maintain ETL solutions and data warehouse structures using SSIS and SQL Server. Build and support SSAS models, optimize ETL performance, ensure data quality, partner with stakeholders, and produce technical documentation to support BI and reporting initiatives.

Develop and manage scalable ETL/ELT pipelines and data warehouses, ensuring performance, reliability, integrity, and consistency. Monitor systems, troubleshoot issues, collaborate with technical and business teams, document architectures, and improve data infrastructure for scalability. The role requires production-quality Python, AWS experience, strong debugging skills, and approximately four years of fintech or startup experience. It is based in Palo Alto with three days per week in the office.

Designs, builds, and maintains scalable Databricks and PySpark data pipelines; develops Python and SQL transformations; identifies insights from ambiguous datasets; and ensures data quality, governance, documentation, and reliable delivery. Collaborates with stakeholders and cross-functional teams to integrate data into applications and dashboards, troubleshoot pipeline issues, and support a Department of Defense Navy client.

Develop and operationalize data applications using Palantir Foundry for national security clients. Translate stakeholder needs into technical solutions, ingest and transform data, build workflows and applications, support scalable data platforms, and lead analytics projects. The role also involves advising clients on data-driven mission impacts, collaborating with Agile teams, and applying DevSecOps, AI/ML integration, security, and compliance practices.

Build and operationalize data pipelines and solutions supporting AI, machine learning, and business intelligence. Responsibilities include designing data solutions, analyzing and cleansing data, implementing governance and security practices, testing transformations, resolving defects, and collaborating across teams. The role requires Python, SQL, AWS, Databricks, Terraform, API design, and AI platform experience.

Builds and maintains scalable batch and streaming data pipelines supporting analytics, machine learning, and business operations. Responsibilities include writing Python, SQL, and Spark code; ensuring data quality and reliability; monitoring and troubleshooting pipelines; implementing testing and observability practices; documenting data architecture; and collaborating with analysts, data scientists, product managers, and engineering teams in an agile environment.

Build and own Dexory’s company-wide data foundation, integrating operational, product, commercial, contractual, and telemetry data. Define trusted data models, business metrics, SLAs, and KPIs; establish secure queryable datasets and AI-assisted natural-language access. Own pipeline health, data quality, architecture, and build-versus-buy decisions while partnering with engineering teams. As the founding data hire, set standards, deliver early value, and help recruit and lead the future data function.

Build and maintain data pipelines in Azure Synapse using Spark, Python, SQL, and C#. Apply data warehouse and lakehouse modeling standards, collaborate with analysts and testing teams, troubleshoot data issues, and develop machine learning and regression analysis capabilities using Azure ML and related tools.

Build and operate Pivotal’s flight data warehouse and telemetry pipelines using AWS Glue, Athena, SQL, and Python. Design curated data marts, fleet health alerting services, data quality gates, dashboards, and query interfaces. Own data infrastructure, deployment, monitoring, and schema evolution end to end while collaborating with firmware, GNC, battery, flight operations, systems engineering, and MRO teams on telemetry and airworthiness decisions.

Build and operate core data infrastructure for edge and hybrid environments, including ingestion, OT/IoT integrations, real-time streaming, data contracts, schemas, validation, lineage, observability, telemetry, and scalable query access. The role requires production ownership, reliability practices, customer integration, and collaboration across engineering and customer-facing teams. Candidates should have 6+ years of experience with production data, streaming, or integration systems and proficiency in Python and/or Go.