Top Data Engineer Jobs

Lead the design and implementation of enterprise data lake architecture for DoW projects. Administer Databricks environments, optimize Delta Lake and ETL pipelines, integrate AI/ML workflows, and develop secure cloud solutions across AWS, Azure, and GCP. Lead technical teams, establish DevSecOps and CI/CD practices, address architecture issues, and coordinate between clients, business analysts, and developers. The role requires TS/SCI clearance and expertise in cloud security, RBAC, compliance, and data governance.

Designs and develops complex cloud-native data solutions, scalable batch and real-time pipelines, APIs, data models, and enterprise data platforms. Implements data governance, quality, lineage, security, observability, CI/CD, automated testing, and operational reporting capabilities. Collaborates with architects, product owners, analysts, governance, security, and platform teams on technology strategy and cloud modernization. Mentors engineers, leads technical design discussions, presents recommendations, and supports AI-ready data products and emerging development tools.

Architect and operate enterprise-scale data platforms supporting cybersecurity operations. Build batch and streaming pipelines, data lakes, warehouses, storage backbones, normalization frameworks, schemas, and telemetry integrations. Optimize scalability, reliability, cost, security, and data quality while enabling SIEM/XDR analytics. Collaborate with security operations, detection engineering, threat management, and data science teams, and maintain architecture documentation and operational runbooks.

Build and maintain production data ingestion, transformation, and pipeline systems for a governed federal data and AI platform. Develop data products, contribute to Terraform infrastructure and CI/CD, monitor quality and reliability, troubleshoot issues, and apply governance and access controls. Support data science and ML teams while operating within FedRAMP, NIST 800-171, and CUI requirements. Collaborate with engineers and document pipelines and processes.

Build enterprise data models, semantic layers, metadata, SQL transformations, and cloud data pipelines. Improve text-to-SQL and generative BI accuracy by partnering with AI engineers on RAG, agents, prompts, testing, and guardrails. Translate business needs into data requirements, validate KPIs, document schemas, apply quality and governance standards, and collaborate with technical and non-technical stakeholders across pharma, life sciences, manufacturing, and healthcare projects.

Designs and deploys data science pipelines, predictive models, retraining systems, and decision-support tools. Leads process improvement initiatives using Lean and Six Sigma, monitors KPIs, and manages operational data stewardship, governance, and integrity. Implements CI/CD, localizes data models, supports Management of Change and project commissioning, and collaborates with senior leadership on analytics strategy and budgets. The role involves office and industrial terminal environments, including chemical exposure, confined spaces, elevated work, and PPE requirements.

Owns the architecture and technical direction of Coalition’s enterprise data platform. Builds scalable Snowflake and dbt infrastructure, governed data models, batch and streaming pipelines, data products, semantic layers, observability, security controls, and AI/ML data foundations. Partners across engineering, product, underwriting, actuarial, security, and business teams while mentoring engineers and establishing technical standards.

Build and maintain secure data-ingestion pipelines, relational databases, ETL workflows, streaming integrations, and scalable data platform services. Support Kubernetes, Terraform, GitLab CI/CD, AWS, PostgreSQL, Kafka, and visualization applications. Improve performance, security, data lineage, and schema-aligned ingestion across structured and unstructured data. Mentor junior engineers and contribute to technical standards and operational improvements in secure enterprise environments.

The Staff Data Reliability Engineer ensures data integrity, quality, freshness, and availability across enterprise pipelines. Responsibilities include defining data SLOs, implementing observability, building resilient Informatica and Python/PySpark pipelines, automating validation and recovery, eliminating operational toil, leading data incidents and postmortems, creating runbooks, and applying AIOps, machine learning, and DataOps practices to detect anomalies and self-heal data workflows.

Own the organization’s data platform, including production pipelines, cloud warehouse architecture, dimensional data models, data quality, testing, documentation, and lineage. Partner with Finance, HR, and Operations stakeholders to build reliable reporting solutions. The role requires independent technical ownership, performance and cost optimization, schema management, and scalable integration of new data sources and business systems.

Build and operate scalable AWS batch and streaming data pipelines, event platforms, data quality frameworks, warehouse models, and ML feature infrastructure. Support personalization, recommendation, AI, experimentation, and model-serving workflows while optimizing Redshift performance. Lead technical architecture, establish engineering standards, mentor engineers, and collaborate with product, analytics, data science, and ML teams.

Designs and maintains scalable Databricks data pipelines on AWS, covering ingestion, transformation, streaming, data quality, and analytics-ready dataset delivery. Builds medallion architecture layers using Spark, Python, SQL, Delta Lake, and AWS services. Responsibilities include source-to-target mapping, unit testing, CI/CD deployment, anomaly analysis, and compliance with FISMA High and multi-tenant governance requirements. Collaborates in Agile teams and uses AI automation tools to accelerate pipeline development and validation.

Lead the architecture, development, governance, and reliability of AssetMark’s Azure and Snowflake data platform. Design scalable ETL/ELT pipelines, write and review Python and SQL, establish engineering standards, oversee DataOps, security, compliance, and FinOps, and integrate AI/ML and GenAI capabilities. Mentor engineers, lead technical design sessions, and partner with Data Science, Product, Security, Compliance, and business stakeholders to deliver governed, production-grade data products.

Design, build, and support enterprise data platforms, semantic/ontology models, and scalable data pipelines. Implement data quality, lineage, metadata, CI/CD, governance (including RLS), and observability. Provide technical leadership, mentor team members, and collaborate with stakeholders to enable analytics, AI, and LLM-driven solutions.

Designs and leads data architecture, models, metadata standards, integration strategies, storage solutions, and virtualized data pipelines. Partners with business stakeholders, technical teams, and product owners to define data requirements and digital ecosystem recommendations. Supports data governance, quality, security, visualization, and analytical systems while guiding technology assessments and digital transformation initiatives.

Leads architecture and engineering standards for Curaleaf’s data and analytics ecosystem. Designs scalable Snowflake platforms, ingestion and transformation pipelines, data models, governance, testing, CI/CD, security, observability, and cost optimization. Partners with engineering, analytics, data science, governance, and business teams to deliver trusted data products. Provides technical direction, mentorship, incident leadership, and cross-functional alignment as a hands-on principal individual contributor.

Designs and operates data infrastructure for a clinical trial platform. Builds pipelines that extract, normalize, validate, and link information from clinical documents and publications into Aurora and GraphDB. Develops embeddings, vector search, RAG workflows, data-serving APIs, lineage tracking, and regulatory audit trails. The role also requires AWS-based orchestration, clinical data quality controls, biomedical knowledge graph modeling, and deployment automation using infrastructure-as-code and containerization.

Designs and maintains scalable data pipelines, warehouse models, and analytics solutions. Builds natural-language data interfaces using GenAI and LLM techniques, optimizes SQL, improves data quality and observability, and supports distributed data platforms on AWS or GCP. Partners with engineering, analytics, business, and product teams, contributes to shared codebases, establishes development standards, and participates in production on-call and incident response.

Build and oversee scalable data platforms, warehouses, lakehouses, pipelines, ETL processes, and dimensional models using Databricks, Snowflake, AWS, Spark, Python, PySpark, and SQL. Integrate tracking data, optimize queries, implement governance and monitoring, and support AI/ML models. Partner with cross-functional teams, deliver dashboards and insights, mentor engineers and analysts, and automate infrastructure deployment using Terraform and CI/CD practices.

Build and operate production data pipelines and enterprise data services supporting AI models, agents, and mission applications. Integrate structured and unstructured data across relational, document, vector, and graph systems; establish data quality, lineage, metadata, governance, and secure access practices; and collaborate with AI, platform, government, and commercial engineering teams. Deploy services in Kubernetes and classified environments while developing reusable ingestion, orchestration, documentation, and operational patterns.

Build and operate production data integrations and ETL pipelines using AWS Glue, PySpark, Python, SQL, and ClickHouse. Responsibilities include designing schemas, integrating unfamiliar data sources, monitoring pipeline health, debugging incidents, improving observability, supporting restricted-environment deployments, and participating in on-call operations. The role collaborates with forward-deployed engineers and requires full-time onsite work in New York City, occasional travel to Arlington, and eligibility for a U.S. government security clearance.

Build and evolve scalable AWS data platforms, enterprise data warehouses, pipelines, models, and integrations. Develop Redshift schemas, implement data quality and observability controls, optimize data jobs, and support lineage and documentation. Partner with data science, analytics, and business teams to deliver reliable datasets. Participate in Agile ceremonies, code reviews, automation, and mentoring while promoting modern, reusable data engineering practices.

Manage the health, performance, reliability, and cost of customer Snowflake and Databricks environments. Responsibilities include customer onboarding, account and warehouse setup, permissions, data ingestion, query optimization, cost reporting, incident response, escalation support, and automation using Python and Terraform. The role also advises customers and internal teams on data platform best practices and participates in on-call rotations.

Supports the enterprise data and analytics platform by building Microsoft Fabric lakehouses, warehouses, pipelines, semantic models, and Power BI reporting. Integrates D365, manufacturing, warehouse, supply-chain, IoT, and operational data; develops SQL and DAX solutions; implements data quality, governance, security, monitoring, and documentation; and troubleshoots production issues. The role also supports AI-enabled analytics and collaborates with business teams, vendors, and technical stakeholders in a manufacturing environment.

Lead the technical direction and production operations of Handshake’s data platform, including Airflow orchestration, ingestion, batch and streaming pipelines, BigQuery, Kubernetes, and cloud infrastructure. Build reliable workflow development and deployment paths, improve data resilience and observability, establish data contracts, support AI workloads, troubleshoot incidents, and mentor engineers across application, analytics, ML, and cloud teams.