Top Data Engineer Jobs in San Diego, CA

Lead design and execution of data migration, archival, and disaster recovery for a cloud-based court case management modernization. Own migration strategy, data lifecycle and retention, DR/HA design and drills, tooling evaluation, documentation, and coordination with Data Engineering, Governance, Security, and stakeholders to ensure auditable, resilient, and compliant migrations.

Design, build, and maintain production-scale data systems and web-scale datasets (Spark/Flink) to support personalization ML. Drive dataset/schema design, collaborate with algorithm teams, and improve distributed big-data processing for member experience.

Design, build, test, deploy, and support enterprise data solutions for financial reporting, reserving, and claims. Develop and maintain pipelines, transformations, data models, and controls using SQL, Python, PySpark, and Microsoft Fabric. Automate reconciliations and data quality checks, partner with business stakeholders, perform testing, review code, mentor peers, and resolve production issues to improve reporting accuracy and operational efficiency.

Technical leader for Data Engineering responsible for designing, operating, and optimizing Snowflake-based data platforms, self-hosted orchestration (Prefect), dbt practices, AWS infrastructure, and data pipelines. Drive architecture, mentor engineers, partner cross-functionally, ensure observability and production-grade practices, and enable model-ready datasets and AI/LLM workflows.

Design, build, operate, and optimize large-scale production data pipelines and foundational Healthcare Map data products using Python, SQL, Airflow, Spark, and AWS. Ensure data quality, observability, and performance; debug complex workflows; contribute to architecture, CI/CD, testing, documentation, and cross-functional delivery to enable analytics and AI/ML use cases.

Design, build, and scale reliable batch and real-time data pipelines and warehouses. Partner with engineering, data science, and business teams to define data models, improve observability, optimize storage and querying, and evangelize data engineering best practices across the company.

Design, build, and maintain scalable Databricks-based data pipelines and ELT workflows; develop and document dbt models; define data modeling methodology; ensure data quality, reliability, and scalability; collaborate with clients and cross-functional teams; provide technical guidance and deliver improvements to clients' data infrastructure and analytics capabilities.

Lead data engineering teams to design, build, and maintain scalable cloud-native data pipelines, data platforms, and data products using Snowflake, Databricks, Airflow, and modern architectures. Optimize ETL/ELT, SQL, and transformations for analytics, reporting, and AI use cases; ensure data quality, governance, observability, and stakeholder alignment. Produce documentation, runbooks, and present technical solutions to business and technical audiences.

Design, build, and optimize scalable batch and streaming data pipelines. Model and process structured and unstructured data, ensure production reliability and observability, apply ML techniques for insight extraction, and drive data strategy while adopting modern tooling and IaC practices.

Lead delivery of enterprise data governance, MDM, and modern data-platform components including Snowflake, CDC pipelines, schema-first contracts (JSON Schema/Avro/Protobuf), data catalog and PII classification, CI/CD schema management, AWS data services, and day-to-day leadership of an offshore delivery team.

Build and own data platform infrastructure and scalable pipelines (ingestion, orchestration, transformation, delivery). Define governance, observability, and AI-first architecture. Optimize performance, ensure data quality, mentor engineers, and integrate AI-assisted tooling.

Own and evolve the data platform: architect, build, and maintain data models, pipelines, and access layers. Ensure data quality, governance, and reliability; monitor and optimize infrastructure; provide technical leadership, code reviews, and collaborate with cross-functional teams to deliver scalable data solutions for operations and analytics.

Build and maintain scalable data pipelines and Python automations for ingesting, processing, and enriching data. Integrate SaaS sources via APIs, use Databricks/PySpark/Delta Lake and AI tools to power lead generation, enrichment, deduplication, and entity resolution. Collaborate cross-functionally to translate business needs into automated, production-ready solutions.

Lead technical design and hands-on development of Pantheon’s data platform, extending ingestion, curation, retention services and ML/LLM infrastructure. Mentor engineers, drive architecture and engineering standards, operate in a DevOps model, partner cross-functionally, and participate in on-call support to ensure reliable, compliant data access and analytics.

The Senior Data Engineer will design and build scalable data platforms, handle automation for data operations, and ensure reliability and performance of data pipelines while collaborating with various teams.

Lead a team to design, build, and maintain cloud data engineering solutions using Microsoft Fabric and Azure Synapse. Responsibilities include architecture, data pipelines, lakehouse and warehousing implementation, governance, performance optimization, mentoring engineers, cross-functional collaboration, support/on-call, documentation, and ensuring reliability and compliance.

Lead the vision and execution for scalable data governance and quality across Airbnb. Architect future-proof data ecosystems, define best practices and tooling, drive cross-functional policy implementation (privacy, legal, infosec), mentor senior engineers, and influence executive strategy to embed stewardship into the data product lifecycle.

Lead architecture, build, and operate Prodege's high-scale data platform including batch, ELT, and near-real-time streaming pipelines. Deliver Medallion-modeled data, governance, lineage, observability, and feature/ML data infrastructure. Drive platform standards, optimize performance and cost, and mentor teams while partnering with product, analytics, and ML groups to enable analytics, experimentation, and AI-driven applications.

Build end-to-end data and analytics features: ingest and model data, build transformations and pipelines, create client-facing dashboards and interfaces, and develop LLM-powered workflows to accelerate analysis and app development. Partner with stakeholders, perform exploratory analyses, and ensure data quality, observability, and maintainability while influencing data engineering strategy.

Design, build, and lead a scalable, configuration-driven data integration platform. Improve performance, automate monitoring and deployments, provide escalated support, and collaborate across product and engineering to deliver reliable, secure data integrations.

Lead architecture, implementation, and optimization of scalable data platforms, ETL/ELT pipelines, and data governance. Ensure security and compliance, mentor junior engineers, drive CI/CD and IaC for data workflows, and collaborate with stakeholders to deliver actionable analytics and reporting.

Design, build, and maintain scalable distributed data pipelines and a secure data lakehouse for streaming and batch processing to support real-time analytics, ML, and experimentation. Automate, test, and harden workflows, architect logical and physical data models, build ML model features, and collaborate with product, analytics, and data science teams to turn data into value.

Build and operate ingestion and transformation pipelines, model and test core warehouse data, enable analytics and reporting, and maintain ML data infrastructure feeding matching and recommendation models. Partner with Engineering, Product, and Data Science to ensure reliable, documented, and trusted data at scale.

Design, build, and maintain scalable ETL/data pipelines across distributed systems. Ensure data quality, integrity, security, and high availability. Monitor and optimize pipeline performance, perform root cause analysis, document systems, collaborate with global teams, and participate in on-call rotations to support production data systems.

Lead architecture and hands-on engineering to design, scale, and govern a cloud-native lakehouse using Databricks and Microsoft Fabric. Build batch/streaming pipelines, ensure HIPAA-compliant PHI handling, produce analytics-ready datasets, mentor engineers, and drive platform strategy, governance, and reliability.