Top Data Engineer Jobs

Design, optimize, and maintain data pipelines and ETL processes using Spark, Apache Iceberg, EMR, and Kubernetes. Ensure data pedigree and security, clean and preprocess data for analytics, collaborate with engineers, data stewards, and mission partners, evaluate and tune data tools, and support administration and continuous improvement efforts.

Design, build, and maintain scalable end-to-end data pipelines and transformation workflows using Databricks, Spark/PySpark, and cloud services. Integrate diverse data sources, implement validation/monitoring, support analytics and reporting, and enable automation, governance, and security in an AWS cloud-native environment.

Develop and maintain Palantir-based ETL and data transformation pipelines using Python and TypeScript. Ingest and format structured and unstructured data, integrate with external APIs, collaborate with data scientists on data models, perform O&M on ETL, produce documentation, conduct tier-2 triage, and design UI elements and custom functions.

Design, build, and maintain large-scale batch and real-time data pipelines on GCP (BigQuery, Dataflow, Dataproc, Pub/Sub). Migrate legacy ETL to cloud-native architectures, optimize query and pipeline performance, enforce data governance and security, and implement automated CI/CD for deployments while collaborating with cross-functional teams.

Design, build, and maintain scalable end-to-end data pipelines using Databricks, Spark, and cloud services. Develop transformation workflows, integrate diverse data sources, enforce data quality, monitoring, and observability. Leverage AWS (S3, Glue, Lambda, Redshift) and cloud warehousing; collaborate with data science, analytics, and business teams. Contribute to governance, security, and automation initiatives.

Design, build, and maintain Azure-based data pipelines and reusable components for reporting and analytics. Integrate, clean, transform, and optimize large-scale streaming and batch data using Databricks/Spark, ADF, and related services. Perform root-cause analysis, own pipeline reliability, and support CI/CD and DevOps practices.

Design, build, and maintain scalable data pipelines and production-grade data products in Palantir Foundry. Implement ETL processes, ensure data quality, optimize pipelines, and collaborate with cross-functional teams. Serve as an independent performer and subject-matter expert, working with unstructured data and open-source/cloud tools for analytics and visualization.

Design and build scalable data architectures and ETL pipelines to support cloud migrations. Develop batch and real-time data processing, ensure data quality and security, monitor and optimize pipelines, troubleshoot migration issues, document processes, and collaborate with cross-functional teams within Agile frameworks.

Design, build, and maintain scalable data pipelines and infrastructure. Integrate large datasets and ML models into production, leverage cloud and big-data frameworks, monitor pipeline health, troubleshoot issues, and collaborate with data scientists and stakeholders while documenting solutions.

Build, maintain, and operate ETL/ELT data pipelines for an enterprise data platform. Support deployment via IaC, monitor and troubleshoot pipelines, run data quality checks, document processes, and collaborate with senior engineers to test and deploy features in a mission-critical government environment.

Design, build, and automate ETL pipelines using Python and SQL; integrate and transform structured and unstructured data from databases, APIs, and files; implement data quality checks, validation, monitoring, and scalable data models; provide Palantir tooling guidance and support for analytics-ready datasets.

Design, build, and maintain scalable GCP data architectures and ETL/ELT pipelines (batch and streaming) using BigQuery, Dataflow, Apache Beam, Cloud Storage, and Pub/Sub. Implement performance tuning, data quality and governance, secure access controls under federal requirements, and automated CI/CD for deployments. Collaborate with data scientists and infrastructure teams to modernize legacy systems and optimize data models for mission use.

Design, build, and deploy production-grade data and AI solutions on Databricks. Create scalable data pipelines, APIs, and Databricks Apps, integrate LLMs/AI APIs, deploy to cloud, implement CI/CD and governance (Unity Catalog), debug full-stack issues, and provide hands-on technical guidance to clients.

As a Senior Lead Data Engineer, you will design, develop, and support data solutions, optimize systems, and collaborate within Agile teams to leverage technology for business transformations.

Design, implement, and administer operational data flows: ingest, clean, enrich, transform, secure, and move mission-critical data. Perform exploratory data analysis, troubleshooting, and collaborate with customers to ensure compliant, reliable data pipelines.

Build and operate scalable data ingestion, transformation, and connector frameworks; design and maintain a knowledge-graph-based data fabric; normalize and vectorize enterprise data for LLM/AI workflows; implement governance, lineage, access controls, and secure APIs to serve ML/agent pipelines.

Design, build, and support scalable data platforms and pipelines (ETL, warehousing, APIs) using DBT, Snowflake, and SQL. Collaborate with architecture, security, and application teams, build CI/CD, provide escalated on-call support, mentor staff, and deliver data solutions for the bank.

The Data Engineer will build and maintain ETL/ELT pipelines, set up a data warehouse, ensure data quality, and collaborate with stakeholders to create usable datasets.

As a Data Engineer, you will design, maintain data pipelines, collaborate with stakeholders, and optimize data solutions across various departments.

Design and maintain scalable ETL/ELT pipelines, develop data solutions using Databricks and Snowflake, and collaborate with cross-functional teams to ensure data quality and performance optimization.

Looking for a Data Engineer focused on data visualization with experience in handling real-world data and effective communication through data. Extra credit for skills in probabilistic classifiers and full-text search engines.

Design, build, deploy, and maintain large-scale data pipelines and ETL using Python and Spark on Azure (Synapse/Databricks). Collaborate with cross-functional teams to translate requirements into data designs, implement CI/CD, monitor and troubleshoot production pipelines, ensure data security and governance, perform code reviews, and document solutions while driving automation and process improvements.

Design, build, and optimize large-scale Databricks/Spark/Delta Lake data pipelines and Redshift data platforms. Implement ETL/ELT patterns, ingest telemetry and behavioral data (Matomo, FullStory, Pendo), improve performance and observability, provide L2/L3 production support and RCA, and mentor engineers while shaping data architecture and best practices.

Serve as a senior technical lead for data engineering consulting: work with customers, design complex data models, write and review efficient code, resolve complex bugs, mentor engineers, lead cross-functional development, and plan/execute automated and performance tests to deliver high-quality data solutions.

Design, build, and optimize scalable Azure-based data pipelines and data warehousing solutions. Implement ETL/ELT, data cleansing, validation, batch and real-time processing, monitor and troubleshoot pipeline performance, ensure data governance and security, document data flows, and collaborate with analysts and stakeholders to deliver reliable analytics-ready data.