Top Senior Data Engineer Jobs

Leads mission data integration and engineering initiatives supporting SOUTHCOM’s Common Intelligence and Operational Pictures. Responsibilities include building ETL pipelines, ingesting and transforming data, managing Databricks, Maven Smart System, and War Data Platform solutions, developing machine learning and statistical models, validating data quality, supporting secure interoperability, collaborating with intelligence and operational teams, delivering training, and documenting analytical processes.

Build and optimize scalable data architectures, pipelines, warehouse tables, and data feeds from diverse sources. Ensure data quality, monitoring, CI/CD reliability, security, and efficient compute and storage usage. Collaborate with engineering, product, and cross-functional teams on data analysis, AI, machine learning, experimentation, and ad technology products. Develop automated data integration and cleansing processes, troubleshoot pipeline failures, and contribute to complex projects involving cloud platforms, MLOps, and large datasets.

Owns customer-facing data engineering engagements from discovery through delivery. Leads workshops, presents to technical and executive stakeholders, translates business needs into data, analytics, and AI solutions, and builds applied AI prototypes. The role also manages scoping, estimates, risks, and cross-functional delivery with Sales, Product, and Engineering. Required expertise includes SQL, cloud data warehouses, Python, dbt, data governance, Git, CI/CD, and strong customer communication.

Design, build, and maintain scalable cloud data pipelines and platform components to power analytics, ML, and applications. Implement transformations, data quality checks, lineage and governance, optimize performance and cost, monitor platform health, and build reusable components across business units while collaborating with product and architecture teams.

Build, deploy, and operationalize complex data pipelines and solutions using AWS, dbt, and ETL tools. Design data architectures, models, transformations, and quality controls; analyze data sources; support AI, machine learning, and business intelligence initiatives; resolve defects and incidents; test data movement and transformation code; and collaborate with teams and business users on data products.

Develop, operate, and maintain distributed parallel streaming and batch data-processing pipelines in classified environments. Responsibilities include configuring data connections, parsing, normalizing, mapping, modeling, enriching, correlating, and deduplicating data; integrating data and analytics SDKs/APIs; applying Agile SDLC practices; maintaining CI/CD pipelines; and updating technical documentation, procedures, and training materials.

Designs, builds, tests, deploys, and maintains secure ETL/ELT pipelines for public health data. Integrates Power Platform, Azure services, APIs, SQL Server, CDC/NIOSH systems, and approved file-transfer platforms. Develops batch, event-driven, and near-real-time data workflows supporting intake, validation, reporting, analytics, notifications, and results delivery while meeting FISMA Moderate, federal privacy, Agile, and DevSecOps requirements.

Design, build, and maintain AWS-based data pipelines supporting genomic data science models. Develop API ingestion, transformation, storage, and delivery workflows across cloud and on-premises environments. Use Python, SQL, AWS services, containers, HPC clusters, and infrastructure-as-code to create scalable solutions. Troubleshoot pipeline, performance, data quality, and modeling issues; implement documentation, testing, monitoring, and RBAC. Collaborate with technical and non-technical stakeholders in a fast-paced consulting environment.

Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.

Lead the architecture and development of Vercel’s next-generation data platform, supporting batch and real-time integrations, analytics, data warehousing, and AI/ML workloads. Design scalable systems using Kafka, ClickHouse, Tinybird, and Snowflake; establish data governance and security standards; guide architectural decisions and roadmaps; collaborate with engineering, product, security, compliance, and leadership teams; write production code; and mentor engineers.

Builds and optimizes scalable data platforms, batch and streaming pipelines, data lakes, warehouses, models, and integration solutions. Leads technical designs, ensures security and governance compliance, owns end-to-end SDLC delivery, troubleshoots production issues, and mentors engineers. Collaborates with engineering, product, analytics, and business stakeholders to deliver reliable data products supporting analytics, machine learning, and business insights.

Senior Data Engineer responsible for monitoring enterprise data pipelines, resolving production incidents, implementing data quality checks, automating operational workflows, and improving reliability across pharmacy data environments. The role also involves AI-enabled process optimization, data governance, documentation, technology evaluation, and operational best practices. Required experience includes data engineering, Azure cloud platforms, Scala or PySpark, SQL, Databricks, Airflow, Azure Data Factory, and production pipeline support.

Builds and operationalizes complex data pipelines and solutions supporting AI, machine learning, business intelligence, and enterprise analytics. Responsibilities include data transformation, cleansing, governance, security, quality, testing, source analysis, defect resolution, and collaboration across teams. The role requires strong Python, SQL, and Databricks expertise, AWS experience, modern software engineering knowledge, and the ability to lead technical teams.

Build and own PFA’s data platform, including healthcare data integrations, ingestion pipelines, data models, quality processes, reporting layers, and analytical infrastructure. Develop production-grade Python and SQL systems, predictive models, AI-enabled workflows, and executive dashboards. Establish engineering standards, testing, governance, and scalable architecture while translating business needs into technical solutions and communicating insights to nontechnical leaders.

Develop, deploy, maintain, and optimize data solutions, including complex SQL queries, stored procedures, ETL/ELT pipelines, and data models. Analyze and refactor legacy processes, perform QA and testing, collaborate across teams, document solutions, and work within Agile environments. The role also supports CI/CD practices and may involve cloud platforms, scalable web data solutions, and business intelligence tools.

Build and maintain data pipelines that support AI/ML applications and national security missions. Design ETL processes, manage data movement, modification, and storage, and ensure scalable data availability. Collaborate with software engineers and data scientists using Databricks, AWS, Python, relational databases, and Agile Scrum practices. Develop solutions for classified environments and help transform complex model outputs into interactive dashboards.

Designs and maintains secure, scalable data pipelines for structured and unstructured sources. Builds segmented data architectures with access controls, classification, governance, lineage, and auditability for sensitive environments. Optimizes workflows across cloud and hybrid platforms, supports production systems, and partners with technical, business, compliance, and cybersecurity stakeholders. The role enables analytics, AI/ML, and digital products while ensuring data integrity, performance, observability, and resiliency.

Designs, builds, and maintains scalable data pipelines and ETL processes. Responsibilities include data modeling, schema design, performance optimization, query tuning, security and compliance, comprehensive logging, error handling, and cross-functional collaboration. The role also guides analysts, engineers, developers, and data consumers while developing scalable platforms for data processing and analytics.

Manage and support PostgreSQL and MySQL databases in a large, high-transaction environment. Own incidents and service requests, meet SLAs, review backups and monitoring reports, respond to alerts, troubleshoot performance issues, and maintain high availability, partitioning, security, and recovery processes. Improve database provisioning automation, monitoring, and alerting while collaborating with engineering teams on recurring issues. The role is hybrid in the Atlanta area.

Administers and supports Azure data platforms, troubleshooting databases, ETL pipelines, large datasets, data-quality issues, and performance bottlenecks. Develops and supports data workflows using Python, Databricks, PySpark, T-SQL, and related Azure technologies. Responsibilities include query optimization, stored procedures, indexing, root-cause analysis, incident ownership, stakeholder communication, production support, and technical coaching.

Build and maintain Databricks-based data platforms, including ingestion, transformation, storage, governance, data modeling, and serving pipelines. Establish medallion architecture standards, canonical data models, quality controls, lineage, schema evolution, and reliable batch or incremental processing. Improve pipeline observability, scalability, idempotency, and recoverability while moving curated data to systems such as ClickHouse. Collaborate across application and analytics teams to create durable, governed production datasets.

Design, build, and maintain Keboola-based data pipelines integrating multiple business systems to deliver accurate, scalable data for supply chain, logistics, and transportation analytics; collaborate with analysts and engineers; support documentation, governance, and pipeline automation.

Designs and maintains enterprise data architecture, pipelines, ETL/ELT workflows, and financial data integrations using Snowflake, AWS, dbt, Informatica, and relational databases. Ensures data quality, develops reporting and dashboards, supports financial planning and analysis, and delivers technology projects end to end. Partners with Finance, Accounting, FP&A, and technology teams, contributes to Agile planning, mentors junior staff, applies AI-assisted tools, and improves data processes and analytical capabilities.

Designs, builds, and optimizes enterprise data pipelines, architectures, and applications supporting financial consolidation, planning, forecasting, reporting, and analytics. Develops ETL/ELT workflows, data models, quality checks, dashboards, and integrations across cloud and relational platforms. Manages projects, partners with Finance and technology teams, improves processes, mentors junior staff, contributes to Agile planning, and evaluates emerging AI and data technologies. Rotational after-hours on-call availability may be required.

Design, build, and operate big-data and streaming platforms (Kafka, Hadoop, Dremio). Develop, deploy, and monitor data pipelines using Java, Python, Spark/Flink; collaborate on data modeling and ingestion, ensure data integrity and availability, act as a Big Data SME, and perform root-cause analysis.