Top Data Engineer Jobs

The Technology Analyst will develop Big Data applications leveraging Hadoop, Java, and Python. Responsibilities include project lifecycle participation, team collaboration, and client interfacing in a global delivery environment.

Lead design, build, and maintain scalable batch and streaming data pipelines and data warehouse models. Ensure data quality, observability, performance, and security. Own workstreams, support production incidents, and enable analysts and downstream applications.

The Senior Data Engineer designs and develops scalable data architectures across platforms, mentors team members, and enhances data engineering capabilities for insights and decision-making.

Design, build, and scale streaming and batch data pipelines and embedding workflows; integrate vector stores and AI platform services; implement observability, data quality, and governance; mentor engineers and lead production-ready AI data foundations supporting ad technology.

Own end-to-end data migrations from messy legacy municipal systems into GovWell: map and ingest data, validate and reconcile with SQL/Python, lead technical onboarding calls with government staff, ensure production data quality, and iterate on tools and playbooks with Product and Engineering.

Design, build, and maintain Databricks and cloud-based data ingestion, storage, and distribution solutions. Implement ETL/warehousing, pipeline orchestration, CI/CD, automated testing, and mentor junior engineers while collaborating across architecture and scrum teams to deliver scalable data products.

Design, build, and lead a scalable, configuration-driven data integration platform. Improve performance, automate monitoring and deployments, provide escalated support, and collaborate across product and engineering to deliver reliable, secure data integrations.

Lead architecture, implementation, and optimization of scalable data platforms, ETL/ELT pipelines, and data governance. Ensure security and compliance, mentor junior engineers, drive CI/CD and IaC for data workflows, and collaborate with stakeholders to deliver actionable analytics and reporting.

Build and maintain reliable Apache Airflow pipelines and workflow engines to move, schedule, and monitor data. Develop automation using Bash, Python, and Java. Work with distributed processing (Spark), containerization (Docker/containerd/Podman), and Git source control to support analytics and operational needs.

Define and lead the technical architecture for a lakehouse on AWS, build production pipelines and data models for time-series and multi-source data, introduce data governance and catalog foundations, author platform playbooks (ADRs, runbooks, Terraform modules), deliver projects end-to-end, and mentor engineers while representing the platform to senior leadership and non-technical stakeholders.

The Staff Data Engineer will build and optimize data pipelines, ensure data reliability, collaborate with teams, and shape the data engineering landscape at CookUnity.

Design, build, and maintain ETL/ELT data pipelines and data models from SQL Server, PostgreSQL, SaaS, and internal apps. Ensure data quality, monitor ingestion, optimize Snowflake performance, collaborate with stakeholders to deliver dashboards and analytics, and recommend data architecture improvements.

Design, build, and operate petabyte-scale data infrastructure: real-time ingestion, storage (open table formats), and serving. Architect scalable Iceberg-based data lakes, optimize Spark pipelines and streaming (Kafka/Flink), drive reliability, cost efficiency, and data contracts; collaborate with product and platform teams and establish best practices and tooling.

Own and build the end-to-end data layer: design high-throughput real-time ingestion, architect petabyte-scale open table formats, implement and optimize Spark-based batch/stream pipelines, enable Kafka/Flink integrations, ensure performance/reliability/cost efficiency, define data contracts and schemas, and establish infrastructure best practices on AWS GovCloud, Kubernetes, and Airflow.

The Senior Data Engineer will manage data tuning on relational databases, utilize Big Data technologies, and coordinate with technical teams and business users.

Lead design and implementation of the enterprise data platform: architect scalable warehouse/lakehouse solutions, build and optimize ETL/ELT pipelines, enforce data quality and governance, tune analytical workloads, and enable reporting and self-service analytics in support of BI and AI initiatives.

Design, build, and operate large-scale AWS-based ETL pipelines using Glue/PySpark to ingest, transform, and curate multi-format data into Apache Iceberg tables. Ensure data quality, metadata management, semantic layer creation (Trino/Athena), CloudFormation deployments, documentation, and compliance within a FedRAMP federal agile environment.

Lead the analytics data layer: design and operate a ClickHouse-backed warehouse, define and validate trusted metrics, build customer-facing React dashboards and internal Grafana/self-serve tooling, and ensure data quality, lineage, and documentation for telemetry-derived insights.

As the Distinguished Architect, you will lead the design and implementation of CloudZero's next-generation data platform, focusing on streaming architecture, data modeling, and optimizing query engines to enhance cost attribution and performance at scale.

Design, build, and maintain data ingestion, ETL, storage, and analytics systems to measure and scale Starlink network performance (including Wi‑Fi). Automate metric aggregation, detect widespread application issues, lead investigations, fuse multi-source data, and apply analytics and ML models to vehicle- and ground-generated data.

Design, build, and iterate data engineering pipelines, dashboards, and reports to support analytics and data science. Partner with stakeholders to translate requirements into engineering solutions, promote self‑service analytics, define best practices, and support implementation using Databricks, Snowflake, and Azure components.

Lead and architect large-scale data products and pipelines across teams. Design and deliver ELT/ETL solutions, evaluate tools, ensure scalability and integrity, influence technology decisions, and research emerging big-data/cloud methodologies (AWS, Hadoop/EMR, Spark, Kafka, Snowflake). Partner with architects and stakeholders to drive implementation and operationalize production workloads.

Design, build, and maintain large-scale data pipelines and APIs for Ventura OS. Optimize data processing workflows, integrate ML models into production, ensure data quality and observability, and collaborate across teams to drive architecture and data infrastructure decisions for streaming and content recommendation use cases.

Design, build, and operate Azure Lakehouse architectures (Databricks, ADLS Gen2, Synapse, ADF); develop and optimize scalable ETL/ELT pipelines and Spark batch/streaming jobs using Python, SQL, and Scala; ensure data quality, performance tuning, cost optimization, production deployments, monitoring, and cross-functional collaboration.

Design, implement, and maintain data analytics platforms, pipelines, dashboards, and ML models for animal nutrition research. Collaborate with researchers, barn managers, and cross-functional teams to collect and analyze farm data, deliver insights and alerts, and develop digital products to improve research, productivity, and sales.