Top Data Engineer Jobs

Build, optimize, and maintain scalable ETL pipelines and data warehousing solutions using Azure technologies. Integrate complex datasets, manage database quality and security, automate processes, document data architectures, and collaborate with data scientists, analysts, and stakeholders. The role also requires experience with SQL, programming languages, artificial intelligence, machine learning, and related analytical methods.

Own Gradial’s end-to-end data platform, including Postgres modeling, multimodal indexing and search, production pipelines, warehouse architecture, governed metrics, and self-service analytics. Establish data quality, contracts, observability, and customer lifecycle instrumentation while setting the platform roadmap and making scalable build-versus-buy decisions. The role requires hands-on technical leadership, strong architecture judgment, and collaboration across engineering, leadership, and go-to-market teams.

Design and implement scalable analytics/data models and dbt pipelines, manage Databricks, build dashboards and BI solutions, partner with business stakeholders, support CRM/marketing analytics, and explore GenAI/ML opportunities to deliver reliable, automated data products.

Build and operate scalable data platforms supporting trading, research, and analytics. Responsibilities include troubleshooting data quality and availability issues, monitoring ingestion pipelines and market data feeds, onboarding datasets, developing ETL/ELT pipelines and APIs, implementing validation and anomaly detection, managing vendors, and improving platform reliability and observability.

Own and evolve production data systems supporting healthcare, claims, billing, analytics, and reporting. Design resilient ingestion patterns, data contracts, models, schemas, and pipelines that handle incomplete, duplicated, delayed, and changing data. Improve performance, observability, failure handling, retries, backfills, and replay safety. Collaborate with engineering, product, operations, and analytics teams while making architecture decisions and documenting technical changes.

Build and support scalable data pipelines, integrations, and lakehouse architecture for marketing, analytics, reporting, and AI applications. Develop data models, monitor data quality, troubleshoot production issues, and provide post-deployment support. Collaborate with analytics, data science, AI, business, and engineering teams to improve data accessibility and long-term platform strategy.

Build and manage data ingestion, integration, transformation, modeling, and quality frameworks for federal financial management and IT programs. Analyze large datasets using SQL, Python, R, PySpark, and Databricks; develop predictive models and apply machine learning techniques. Ensure data governance and quality standards, define Databricks data quality metrics, support client meetings and documentation, and contribute to consulting and business development activities.

Architects, develops, and optimizes scalable data pipelines and cloud-native platforms integrating structured and unstructured data. Leads data engineering teams, establishes engineering standards, ensures data quality, and builds secure enterprise infrastructure supporting analytics, reporting, and mission-critical decision-making. Uses Python or Scala, SQL, Kafka, NiFi, AWS, and distributed systems while collaborating with multidisciplinary teams.

Build, maintain, and improve data integrations that extract, transform, and load retailer data into standardized schemas. Collaborate with engineering, Product, and Client Solutions teams to expand the data platform, improve reliability, monitor outages, and develop new integration capabilities. Work with data pipelines, cloud infrastructure, supply chain concepts, APIs, web scraping, browser automation, and EDI.

Lead data engineering and machine learning initiatives by designing, architecting, developing, and supporting GCP data pipelines. Create information architecture documentation, estimate and plan technical work, collaborate with project teams and business analysts, and translate requirements into technical designs. Develop Python-based ELT pipelines, optimize BigQuery SQL, support CI/CD, and provide technical leadership to projects and teams.

Build, deploy, and maintain scalable data pipelines and platforms for mission-driven projects. Validate and normalize datasets, develop schemas, implement ETL workflows, and create monitoring dashboards and metrics. Collaborate with AI/ML engineers, cloud architects, and technical leaders to support governed, high-quality data flows and reliable model performance. Improve pipeline performance and support data engineering for operational, research, and enterprise systems.

Build and maintain reliable data pipelines, integrations, automations, Airflow workflows, and dbt models. Develop semantic-layer datasets, monitor data quality, troubleshoot pipeline issues, document systems, and collaborate with business stakeholders. The role also involves applying AI, machine learning, and LLM-based tools to data engineering workflows, with potential Power BI and healthcare data work.

Designs and operates secure batch and streaming data pipelines across cloud platforms. Responsibilities include data ingestion, integration, modeling, quality validation, governance, lineage, security, monitoring, workflow orchestration, and performance optimization. The role supports analytics, machine learning, AI-enabled applications, and authoritative data services while documenting architectures, runbooks, and data contracts. Collaboration with engineering, cloud, security, data science, architecture, and customer teams is required, along with contributions to secure data platform proposals.

Build and maintain scalable data pipelines and platforms using SQL, Java, or Python. Acquire, tag, integrate, and process structured, semi-structured, and unstructured data. Apply machine learning models for classification, use BigID and AWS, and collaborate with analysts, engineers, developers, and data consumers in an agile environment. Guide analytical exploration, data examination, platform design, deployment, and maintenance for mission-driven client projects.

Leads the architecture, development, and support of enterprise data warehouses, lakehouses, dimensional models, ETL/ELT pipelines, and governed data products using Microsoft Fabric and related tools. Ensures data quality, security, scalability, performance, and reliability for analytics and reporting. Partners with technical and business stakeholders, documents data solutions, guides architecture decisions, mentors team members, and drives continuous improvement of data engineering standards and practices.

Lead the design, development, optimization, and monitoring of scalable cloud data pipelines using AWS Glue, PySpark, and Amazon Redshift. Integrate diverse data sources, ensure data quality, deploy infrastructure with Terraform, and improve reliability, performance, and cost efficiency. Collaborate with data professionals and stakeholders, document architecture standards, ensure security compliance, mentor junior engineers, conduct code reviews, and evaluate AWS technologies. The role requires a bachelor’s degree, substantial cloud data engineering experience, and hybrid work in specified locations.

Design and optimize Snowflake data warehouses, dbt transformation models, and scalable ETL/ELT pipelines. Develop ingestion utilities, improve SQL and warehouse performance, establish data quality and governance controls, and maintain CI/CD workflows using Git. Collaborate with analysts, data scientists, and business stakeholders across manufacturing, supply chain, and operations. The role also supports modern DataOps practices and may involve orchestration tools and cloud platforms.

Build unstructured-data ingestion and context-layer pipelines for an investment group. Responsibilities include historical backfills, document parsing, entity resolution, deduplication, incremental API synchronization, embeddings, vector and graph storage, access scoping, lineage, PII protection, orchestration, monitoring, cost control, and operational runbooks.

Lead design, development, and operation of large-scale Spark-based data pipelines and lakehouse solutions. Write and review Python/PySpark and Spark SQL production code, optimize Spark workloads on EMR/Databricks, manage Apache Iceberg table operations, drive AI-assisted engineering adoption, automate reliability and CI/CD practices, and lead technical evaluations and communities of practice.

Designs and maintains scalable ETL pipelines, data models, integrations, cloud databases, and healthcare data infrastructure. Ensures security, HIPAA/HITRUST compliance, reliability, monitoring, and documentation while collaborating with product, engineering, and clinical teams. Requires strong SQL and Python expertise, modern data warehousing and orchestration experience, cloud platform knowledge, and familiarity with healthcare interoperability and claims data.

Own and define the technical direction of Dubsado’s data infrastructure as its first dedicated data hire. Assess and improve existing pipelines, warehouse systems, analytical models, reliability, quality, and observability. Build scalable data infrastructure, establish engineering standards and guardrails, support AI and customer-facing data features, and collaborate with engineering, product, and business stakeholders. Over time, expand contributions to backend services, infrastructure, and agentic systems while distributing data ownership across engineering.

Lead Data Engineer responsible for designing, maintaining, and improving data pipelines and platform infrastructure across ingestion, streaming, transformation, and delivery. The role uses Airflow, Spark, Iceberg, Kafka, Flink, dbt, Terraform, and GCP. Responsibilities also include defining data contracts, documenting systems, optimizing performance, reviewing code, supporting access controls and compliance, and collaborating with stakeholders and teammates.

Build and operate the Defense & Systems Data Fabric through metadata ingestion pipelines, API and connector integrations, data-product onboarding, and policy enforcement. Implement policy-as-code and attribute-based access control for export-control and data-sovereignty requirements. Maintain monitoring, alerting, reliability, documentation, and certification workflows across hybrid cloud, on-premises, and edge environments while conforming to architectural and governance standards.

Own and evolve a core data platform from design through production. Build scalable ingestion pipelines, reusable integration frameworks, data models, validation, monitoring, and infrastructure supporting AI-agent and LLM-based data intake. Analyze customer systems and legacy datasets, collaborate on product direction, and maintain reliable backend data infrastructure. The role is fully on-site in San Francisco and may progress toward engineering team management.

Design and operate a large-scale AI/ML data platform spanning ingestion, lakehouse storage, cataloging, governance, vector search, dataset versioning, feature and embedding pipelines, observability, and low-latency retrieval. The role owns production reliability and performance, establishes data standards, supports reproducible model training, manages petabyte-scale storage, and mentors engineers while leading technical architecture.