Top Data Engineer Jobs in San Diego, CA

Leads enterprise-scale data architecture across batch, streaming, warehouse, lakehouse, transactional, AI, and vector-store use cases. Establishes data quality, governance, lineage, observability, access-control, and reliability standards. Defines AI-enabled data engineering patterns, mentors engineers, resolves complex architecture issues, and influences cross-functional roadmaps. Requires hands-on technical leadership and 15+ years of software, data engineering, or architecture experience.

Build and operate governed Finance data infrastructure on Databricks, including scalable pipelines, integrations, semantic layers, data-quality controls, observability, and AI-enabled services. Partner with analytics and enterprise teams on architecture, data contracts, security, recoverability, and production ownership. Optimize platform costs through FinOps practices and use AI coding assistants while validating performance, correctness, and security.

Architects, deploys, and operates scalable data platform infrastructure supporting analytics. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and compliance controls, developing dbt data models, cataloging assets in OpenMetadata, automating with Python, and mentoring engineers. The role requires extensive infrastructure and data engineering experience, cloud expertise, container orchestration, and eligibility for CJIS clearance.

Develop and maintain high-throughput backend and middleware systems for PlayStation’s cloud streaming data platform. Build JVM applications, Flink stream-processing jobs, scalable ETL pipelines, and real-time APIs. Optimize performance, reliability, monitoring, and scalability while supporting Kafka-to-cloud migrations. Collaborate with Business Intelligence and product teams to translate requirements into actionable insights and contribute to data-platform architecture.

Build and support enterprise data ingestion, transformation, modeling, and integration solutions using Microsoft Fabric and Azure data services. Develop batch, API, file, database, and event-driven pipelines; create dimensional and curated data models; write and optimize SQL; resolve data-quality and pipeline issues; and support production workloads. Collaborate in Agile engineering processes using GitHub, Azure DevOps, CI/CD, code reviews, and reusable development standards.

Designs and delivers large-scale, governed data pipelines and infrastructure across the full data lifecycle. Responsibilities include coding in Python and SQL, optimizing Airflow, dbt, AWS, Kafka, and Iceberg/Parquet workflows, implementing data quality and observability, contributing to architecture and tool evaluations, mentoring engineers, collaborating with stakeholders, and participating in on-call support.

Build and maintain scalable ETL/ELT pipelines, integrate structured and unstructured data, create curated datasets, and enforce data quality, lineage, governance, security, and privacy controls. Develop data models, documentation, and NLP-ready data while supporting visualization, analytics, and MLOps deployments. Collaborate with data scientists, clinicians, government personnel, and program staff to translate analytical needs into reliable data solutions.

Build and maintain scalable ETL/ELT pipelines, integrate structured and unstructured government data, and produce curated datasets for analytics and machine learning. Ensure data quality, lineage, governance, security, and privacy compliance while documenting models and transformations. Support NLP, LLM, visualization, and MLOps initiatives; monitor workloads, resolve pipeline failures, and provide technical guidance to data scientists and government stakeholders.

Design and maintain scalable AWS-based data pipelines, architectures, ETL/ELT workflows, databases, and big data solutions. Integrate and transform complex datasets using Python, SQL, Spark, Hadoop, and Kafka. Build data models, dashboards, reports, and analytics solutions while implementing data quality, governance, security, monitoring, and compliance processes. Support migrations and modernization projects, collaborating with data scientists, analysts, and business stakeholders to deliver reliable, decision-ready data.

Design and modify data solutions using Databricks and Palantir Foundry; build data models and reports from ERP and legacy-system datasets; define and execute tests; and translate stakeholder reporting needs into technical specifications. The role requires SQL, Python, PySpark, database architecture, cloud architecture, software engineering practices, independent execution, documentation, and collaboration. It supports an Army client in a hybrid Arlington, Virginia environment and requires an active Secret clearance.

Maintains and monitors Azure Databricks data pipelines, troubleshoots data issues, collaborates with stakeholders on data requirements, documents processes, performs backups and disaster recovery drills, and supports data access and usage. The role requires extensive data engineering experience, Azure expertise, incident management knowledge, and strong problem-solving skills.

Build and maintain reliable data pipelines, computed tables, and scalable data models using SparkSQL and PySpark. Integrate IoT, product, customer, external, unstructured, and ML-related data into a central data lake for analytics, causal inference, model training, and dashboards. Ensure high data quality and uptime while collaborating with data science, analytics, AI/ML, and engineering teams.

Lead the reverse-engineering and migration of legacy ETL pipelines from SAP BusinessObjects Data Services and JD Edwards into dbt and Databricks architectures. Build automated data validation, maintain SQL Server write-back mechanisms, collaborate with client stakeholders, and lead and mentor a small engineering team. The role requires hands-on expertise in SQL, Python, Spark, orchestration, cloud platforms, and legacy ERP and ETL migrations.

Designs, builds, tests, and maintains scalable data infrastructure and ETL/ELT pipelines for mission partners. Uses AWS Glue, Glue Catalog, and Step Functions to integrate, migrate, transform, and validate data. Develops data models, schemas, queries, and production-ready datasets while collaborating with analytics and business teams. The role may also support AI/ML pipelines, data governance, DevOps workflows, Databricks, and real-time streaming architectures.

Architects, builds, tests, and maintains scalable data infrastructure and ELT/ETL pipelines using AWS Glue, Glue Catalog, and Step Functions. Performs data modeling, SQL querying, data migration, schema optimization, validation, and data quality improvements. Partners with analytics and business teams to transform raw data into production-ready datasets. The role may also support AI/ML pipelines, governance, access controls, DevOps workflows, and real-time data architectures.

Own Eve’s data platform end to end, including ingestion, orchestration, incremental dbt models, Snowflake administration, Terraform infrastructure, observability, access controls, and pipeline reliability. Build schema-change detection, freshness SLAs, CI environments, and incident processes while managing performance and compute costs. Partner with analytics engineers and stakeholders to maintain trusted data for reporting and AI systems, using AI-assisted development and documenting platform standards.

Build and deploy scalable data pipelines and platforms that organize structured and unstructured data. Perform ETL, pipeline integration, analytical exploration, and data platform design and maintenance in agile, multidisciplinary teams. Develop deployable software, support data visualization, and work with relational and non-relational databases. Projects support missions including fraud detection, cancer research, and national intelligence.

Build and deploy scalable data pipelines and platforms that organize structured and unstructured data for mission-driven clients. Collaborate with analysts, developers, and data consumers in agile teams while supporting data platform assessment, design, development, maintenance, analytical exploration, and data examination. Apply ETL, software development, visualization, database, automation, and cloud technologies to support applications including fraud detection, cancer research, and national intelligence.

Designs, builds, operates, and modernizes secure ETL/ELT data pipelines for public health interoperability. Responsibilities include using NiFi, Kafka, Python, and SQL; deploying pipelines on Azure or AWS; implementing CI/CD with GitHub Actions; monitoring with Grafana; troubleshooting production systems; supporting legacy migrations; and ensuring data security, compliance, scalability, and reliability. The role collaborates with architects, integration engineers, data scientists, DevOps/SRE, and product teams.

Entry-level rotational Data Engineer responsible for SQL development, SSIS data transformations, data validation, testing, performance tuning, data profiling, and reporting solutions. The role collaborates with IT, project managers, analysts, and business stakeholders to implement enterprise data solutions across Salesforce, Marketing Cloud, and other platforms. This is a 100% on-site position in San Diego, California.

Leads data architecture and engineering modernization across business domains. Designs and implements complex pipelines, integrations, data models, and orchestration using Snowflake, AWS, and Airflow. Establishes data quality, governance, security, testing, observability, CI/CD, and reliability practices. Provides architectural guidance, evaluates technical solutions, leads cross-team initiatives, mentors engineers, and improves platform performance and cost efficiency.

Build and maintain production ETL pipelines for structured and unstructured data, integrate machine learning models for enrichment, develop data transformations and quality monitoring, design warehouses and dimensional models, establish governance frameworks, and contribute to data architecture and platform improvements. Collaborate with data scientists, analysts, and engineering teams to deliver reliable, actionable intelligence.

Leads the architecture, development, and production operation of complex data pipelines and platforms. Resolves scalability, data quality, reliability, and architectural challenges; evaluates data technologies; mentors engineers; establishes engineering standards; partners with product, analytics, and operations teams; communicates technical trade-offs; and conducts proofs of concept to guide platform strategy.

Designs and optimizes Power BI dashboards, semantic models, datasets, DAX measures, and reporting solutions for identity governance and administration analytics. Partners with ETL engineers and stakeholders to validate Parquet data from Azure Synapse and SailPoint sources, establish data-quality checks, monitor refreshes, define KPIs, and document data lineage. Supports compliance reporting, access governance, audit readiness, certification metrics, and Zero Trust analytics.

Build and maintain scalable Snowflake-based ELT/ETL pipelines, production data models, and observability frameworks. Translate business requirements into reliable solutions supporting analytics, BI, and ML. Establish data quality, governance, lineage, security, and compliance practices while optimizing warehouse performance, cost, and data freshness. Collaborate with Analytics, Product, and Engineering teams, contribute to architecture decisions, evaluate Snowflake tooling, and use AI-assisted development tools to accelerate delivery.