Data Engineer Jobs in Sunnyvale, CA

Build and operate a security data platform connecting SIEM, identity, asset, threat intelligence, and knowledge graph systems. Design core APIs, distributed data pipelines, schemas, risk scoring, entity resolution, and coverage queries. Own architecture and platform workstreams from scoping through delivery, collaborate with detection and identity teams, review code, mentor engineers, and drive partner adoption. The role requires expert Python, deep Splunk expertise, cybersecurity experience, and senior-level project ownership.

Designs and deploys scalable data solutions supporting AI agents, LLM workflows, and autonomous decision-making. Builds cloud-based pipelines, semantic layers, data models, monitoring, governance, and full-stack services. Partners with engineering, AI/ML, product, and business teams to deliver reliable, traceable, high-performing data products and enterprise applications. Requires expertise in distributed data processing, cloud platforms, databases, APIs, DevOps, and modern frontend and backend development.

Designs and operates billion-scale data platforms supporting streaming, batch, advertising, marketing, measurement, and machine learning workloads. Builds reliable, tenant-aware pipelines with replayability, fault recovery, observability, quality controls, and cost efficiency. Leads domain architecture, performance optimization, design reviews, mentoring, incident response, and governed automation across global advertising products.

Design and implement high-performance, scalable storage systems and client libraries for AI workloads. Build data loading, checkpointing, caching, POSIX-style and object-store integrations, observability and telemetry, and validate performance with platform, SRE, and AI teams using modern engineering practices.

Design and implement cloud-native data management services (catalog, metadata, dataset/checkpoint lifecycle) for exabyte-scale GPU workloads. Build backend systems on Kubernetes and major cloud providers, collaborate with research and cross-functional teams, document architecture, and drive integration with storage/compute innovations (GPU Direct Storage, DPU).

Design, build, secure, and operate Microsoft’s clean room data platform and infrastructure. Develop scalable data onboarding, identity resolution, audience activation, measurement, and partner collaboration capabilities. Implement access controls, encryption, network isolation, monitoring, auditing, compliance, and reliability standards for sensitive data environments. Automate provisioning, deployments, and operational controls while leading incident investigations and platform improvements. Partner with engineering, privacy, security, marketing, data engineering, and external organizations on long-term privacy-safe data collaboration solutions.

Develop large-scale data pipelines, infrastructure, and tools for generating, analyzing, and evaluating machine learning datasets from autonomous driving logs and simulation data. Build robust data platforms supporting Waymo Driver launch evaluations, collaborate with Data Science and Quantitative Analytics teams, and improve dataset curation, sampling, slicing, accuracy, and completeness.

Designs and builds scalable microservices, distributed applications, ETL pipelines, and AI data workflows for autonomous vehicle data. Responsibilities include streaming, orchestration, security, data management, model deployment, RAG workflows, video curation, behavioral search, and optimization of large-scale distributed systems. The role collaborates with software and AI experts while using technologies such as Python or Golang, Kubernetes, Hive, Parquet, SQL, vector databases, Spark, and NVIDIA RAPIDS.

Leads an engineering team building scalable ML data infrastructure, including data ingestion, processing, storage, access, pipelines, and analytical platforms. Defines architecture, evaluates technologies, improves performance and reliability, translates data requirements into technical solutions, and ensures governance, security, and regulatory compliance. Provides mentorship, strategic roadmap input, and cross-functional collaboration with ML engineers, data scientists, platform engineers, and product teams.

Build and operate Block’s data platform, including warehousing, orchestration, business intelligence, governance, and agentic workflows. Develop reliable Python services, developer tools, and Terraform infrastructure for petabyte-scale data. Partner with data practitioners and product teams, establish engineering standards, provide technical direction, and mentor engineers while ensuring security, reliability, observability, usability, and data governance.

Build and operate a scalable data platform supporting warehousing, orchestration, business intelligence, governance, and agentic workflows. Develop Python services, Terraform infrastructure, and developer tools for petabyte-scale data. Partner with data practitioners and product teams, establish security and reliability patterns, set technical direction, improve engineering standards, and mentor engineers.

Leads hands-on design, development, and optimization of AI data movement and distributed storage systems. Responsibilities include integrating NVIDIA NIXL and DDN Infinia with GPU inference platforms, optimizing GPU-to-storage I/O using GPUDirect Storage, RDMA, and NVMe-over-Fabrics, developing KV cache and multi-tier storage strategies, benchmarking production systems, resolving performance bottlenecks, influencing distributed inference architecture, and mentoring engineers.

Architects, builds, and scales data pipelines, ETL processes, data models, and platforms using dbt, Snowflake, Airflow, and Fivetran. Leads data engineering best practices including CI/CD, data quality, governance, and observability. Collaborates with Data Science, Engineering, Product, and business stakeholders; mentors engineers and data scientists; contributes to architecture and technical decisions; and participates in on-call support.

Develop and manage enterprise data platforms, pipelines, APIs, MDM/RDM solutions, governance frameworks, and AI-enabled observability for semiconductor supply chain data. Integrate SAP, PLM, planning, and business systems while building canonical models, metadata catalogs, lineage, data quality processes, and agentic AI workflows. Partner with engineering, business, and IT stakeholders to deliver scalable, governed data solutions across manufacturing and supply chain operations.

Build and optimize edge and cloud data pipelines for perception ML models. Scale data engine for high-fidelity labels, reduce annotation costs, integrate foundation models for automated labeling and QA, leverage software/hardware-in-the-loop testing, and support DoD field use cases and model deployment lifecycle.

Build and manage high-volume analytics and production ETL/ELT systems. Partner with Data Science, Engineering, and Product teams to ensure reliable Search data logging and usage. Own data quality, observability, security, compliance, SLAs, scalability, performance, and cost efficiency across the data lifecycle. The role requires expertise in Python, SQL, data modeling, distributed data technologies, massive datasets, and data pipeline optimization.

Build and operate reliable data pipelines transforming flight test, simulation, and operational data into versioned, traceable ML training datasets. Responsibilities include ingestion, normalization, time alignment, validation, dataset lineage, anomaly mining, labeling workflows, deployment feedback loops, monitoring, and data coverage analysis. Collaborate with flight test and operations teams on instrumentation and data access. The role is on-site in Boston and requires production data engineering experience supporting machine learning pipelines.

Own production-grade analytics pipelines from design through maintenance, build dbt-based analytical models using Kimball dimensional modeling, and deliver reliable data features. Partner with Product, Engineering, and Data Science teams to translate retail signals into business insights. Establish standards for monitoring, documentation, reproducibility, and code quality. Use AI tools to accelerate development and prototype conversational data experiences for natural-language retail data exploration.

Own and scale the company’s analytics infrastructure, including BigQuery data warehouses, dbt data marts, external data exchange, DataOps, data quality, governance, security, and cloud cost optimization. Partner with application database teams, analytics engineers, and data scientists to enable reliable metrics and scalable client data delivery.

Build and consolidate data infrastructure for ML training: design automated ETL/ELT pipelines, implement DataOps practices (validation, monitoring, anomaly detection), integrate annotation workflows, manage dataset versioning and storage, and collaborate with ML engineers and operations to produce high-quality, reproducible training datasets.

The Data Engineer will analyze data quality, design prediction algorithms, collaborate with the engineering team, and generate business insights.

Design and implement data platforms, manage data infrastructure, develop streaming architectures, improve performance, and ensure data compliance.

Lead architecture and standards for CoreWeave's enterprise data ecosystem: design lakehouse/streamhouse architectures, modeling and semantic layers, governance, data quality, lineage, observability, and reusable data products. Drive cross-domain solutions, resolve performance and scalability constraints, evaluate core data technologies, conduct architecture reviews, and mentor senior engineers.

Hybrid hardware/software engineer responsible for characterizing 4D LiDAR sensors, supporting customer integrations, building production Python/C++ data pipelines, running calibration and field debugging, maintaining ETL for high-volume sensor data, and coordinating data labeling and collection operations.

Own the architecture and operation of large-scale autonomous-driving datasets, from multimodal vehicle-data ingestion through curation, quality monitoring, labeling, versioning, and ML training delivery. Design schemas, storage, query, distributed pipelines, observability, and reproducible dataset splits. Lead data-quality, drift, anomaly, auto-labeling, human-review, and closed-loop workflows, including trajectory preparation for imitation learning and offline reinforcement learning. Provide staff-level technical leadership across autonomy, annotation, storage, governance, and model teams.