Top Senior Data Engineer Jobs

Design, build, and maintain AWS-based data pipelines supporting genomic data science models. Develop API ingestion, transformation, storage, and delivery workflows across cloud and on-premises environments. Use Python, SQL, AWS services, containers, HPC clusters, and infrastructure-as-code to create scalable solutions. Troubleshoot pipeline, performance, data quality, and modeling issues; implement documentation, testing, monitoring, and RBAC. Collaborate with technical and non-technical stakeholders in a fast-paced consulting environment.

Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.

Lead the architecture and development of Vercel’s next-generation data platform, supporting batch and real-time integrations, analytics, data warehousing, and AI/ML workloads. Design scalable systems using Kafka, ClickHouse, Tinybird, and Snowflake; establish data governance and security standards; guide architectural decisions and roadmaps; collaborate with engineering, product, security, compliance, and leadership teams; write production code; and mentor engineers.

Builds and optimizes scalable data platforms, batch and streaming pipelines, data lakes, warehouses, models, and integration solutions. Leads technical designs, ensures security and governance compliance, owns end-to-end SDLC delivery, troubleshoots production issues, and mentors engineers. Collaborates with engineering, product, analytics, and business stakeholders to deliver reliable data products supporting analytics, machine learning, and business insights.

Senior Data Engineer responsible for monitoring enterprise data pipelines, resolving production incidents, implementing data quality checks, automating operational workflows, and improving reliability across pharmacy data environments. The role also involves AI-enabled process optimization, data governance, documentation, technology evaluation, and operational best practices. Required experience includes data engineering, Azure cloud platforms, Scala or PySpark, SQL, Databricks, Airflow, Azure Data Factory, and production pipeline support.

Builds and operationalizes complex data pipelines and solutions supporting AI, machine learning, business intelligence, and enterprise analytics. Responsibilities include data transformation, cleansing, governance, security, quality, testing, source analysis, defect resolution, and collaboration across teams. The role requires strong Python, SQL, and Databricks expertise, AWS experience, modern software engineering knowledge, and the ability to lead technical teams.

Build and own PFA’s data platform, including healthcare data integrations, ingestion pipelines, data models, quality processes, reporting layers, and analytical infrastructure. Develop production-grade Python and SQL systems, predictive models, AI-enabled workflows, and executive dashboards. Establish engineering standards, testing, governance, and scalable architecture while translating business needs into technical solutions and communicating insights to nontechnical leaders.

Develop, deploy, maintain, and optimize data solutions, including complex SQL queries, stored procedures, ETL/ELT pipelines, and data models. Analyze and refactor legacy processes, perform QA and testing, collaborate across teams, document solutions, and work within Agile environments. The role also supports CI/CD practices and may involve cloud platforms, scalable web data solutions, and business intelligence tools.

Build and maintain data pipelines that support AI/ML applications and national security missions. Design ETL processes, manage data movement, modification, and storage, and ensure scalable data availability. Collaborate with software engineers and data scientists using Databricks, AWS, Python, relational databases, and Agile Scrum practices. Develop solutions for classified environments and help transform complex model outputs into interactive dashboards.

Designs and maintains secure, scalable data pipelines for structured and unstructured sources. Builds segmented data architectures with access controls, classification, governance, lineage, and auditability for sensitive environments. Optimizes workflows across cloud and hybrid platforms, supports production systems, and partners with technical, business, compliance, and cybersecurity stakeholders. The role enables analytics, AI/ML, and digital products while ensuring data integrity, performance, observability, and resiliency.

Designs, builds, and maintains scalable data pipelines and ETL processes. Responsibilities include data modeling, schema design, performance optimization, query tuning, security and compliance, comprehensive logging, error handling, and cross-functional collaboration. The role also guides analysts, engineers, developers, and data consumers while developing scalable platforms for data processing and analytics.

Manage and support PostgreSQL and MySQL databases in a large, high-transaction environment. Own incidents and service requests, meet SLAs, review backups and monitoring reports, respond to alerts, troubleshoot performance issues, and maintain high availability, partitioning, security, and recovery processes. Improve database provisioning automation, monitoring, and alerting while collaborating with engineering teams on recurring issues. The role is hybrid in the Atlanta area.

Administers and supports Azure data platforms, troubleshooting databases, ETL pipelines, large datasets, data-quality issues, and performance bottlenecks. Develops and supports data workflows using Python, Databricks, PySpark, T-SQL, and related Azure technologies. Responsibilities include query optimization, stored procedures, indexing, root-cause analysis, incident ownership, stakeholder communication, production support, and technical coaching.

Build and maintain Databricks-based data platforms, including ingestion, transformation, storage, governance, data modeling, and serving pipelines. Establish medallion architecture standards, canonical data models, quality controls, lineage, schema evolution, and reliable batch or incremental processing. Improve pipeline observability, scalability, idempotency, and recoverability while moving curated data to systems such as ClickHouse. Collaborate across application and analytics teams to create durable, governed production datasets.

Design, build, and maintain Keboola-based data pipelines integrating multiple business systems to deliver accurate, scalable data for supply chain, logistics, and transportation analytics; collaborate with analysts and engineers; support documentation, governance, and pipeline automation.

Designs and maintains enterprise data architecture, pipelines, ETL/ELT workflows, and financial data integrations using Snowflake, AWS, dbt, Informatica, and relational databases. Ensures data quality, develops reporting and dashboards, supports financial planning and analysis, and delivers technology projects end to end. Partners with Finance, Accounting, FP&A, and technology teams, contributes to Agile planning, mentors junior staff, applies AI-assisted tools, and improves data processes and analytical capabilities.

Designs, builds, and optimizes enterprise data pipelines, architectures, and applications supporting financial consolidation, planning, forecasting, reporting, and analytics. Develops ETL/ELT workflows, data models, quality checks, dashboards, and integrations across cloud and relational platforms. Manages projects, partners with Finance and technology teams, improves processes, mentors junior staff, contributes to Agile planning, and evaluates emerging AI and data technologies. Rotational after-hours on-call availability may be required.

Design, build, and operate big-data and streaming platforms (Kafka, Hadoop, Dremio). Develop, deploy, and monitor data pipelines using Java, Python, Spark/Flink; collaborate on data modeling and ingestion, ensure data integrity and availability, act as a Big Data SME, and perform root-cause analysis.

Administer, architect, optimize, secure, and automate PostgreSQL, MySQL, and Snowflake databases in Azure. The role develops resilient database solutions for healthcare applications, manages performance, access controls, encryption, backups, and disaster recovery, and creates technical documentation. It collaborates across Scrum teams, presents solutions to stakeholders, evaluates emerging technologies, uses approved AI tools, solves complex database challenges, and mentors junior engineers.

Designs, implements, and operates data management systems supporting defense intelligence needs. Develops data architectures, structures, and integration strategies; optimizes data storage, throughput, and query performance; collaborates with data users and intelligence community members; and contributes to complex analytical products and information-sharing efforts.

Develop and optimize enterprise Oracle SQL and PL/SQL solutions, including data models, database designs, APIs, ETL processes, batch jobs, and performance tuning. Collaborate with cross-functional teams to translate requirements into scalable implementations, support testing and CI/CD, troubleshoot production issues, and document solutions. Provide technical leadership, mentor engineers, participate in architecture and code reviews, and drive engineering best practices across data modernization initiatives.

Build and maintain production-grade ETL/ELT pipelines across legacy systems, AWS, Azure, and Snowflake. Develop feature stores, optimized datasets, and vectorization pipelines supporting machine learning and generative AI. Design cross-platform orchestration, data quality, observability, and governance frameworks while optimizing performance and cloud costs. Partner with MLOps, data science, infrastructure, security, and database teams to deliver scalable AI-ready data solutions.

Manage data engineering and analytics initiatives spanning SQL analysis, Power BI dashboards, dimensional data modeling, cloud data warehouses, and ETL pipelines. Partner with business stakeholders to translate questions into actionable insights and communicate findings clearly. Apply AI-assisted tools and evaluate LLM-based analytics capabilities such as natural-language querying, automated insight generation, and anomaly detection.

Build and scale batch and streaming data pipelines using Databricks, Spark, Kafka, AWS, Python, SQL, dbt, and Fivetran. Develop ETL/ELT and reverse ETL workflows, curated datasets, dashboards, data quality controls, and data activation integrations with Salesforce, HubSpot, and Braze. Partner with Finance, Marketing, Product, and Engineering on reporting, forecasting, segmentation, and AI/ML applications, including RAG, embeddings, and agent-based systems.

Design, build, and operate scalable ETL/ELT pipelines and backend data services using Go. Integrate relational, NoSQL, blockchain, API, web, and social data sources; develop NLP pipelines for text extraction and classification; and support production APIs, dashboards, alerts, and intelligence systems. Collaborate with engineering, data science, and investigations teams on architecture, data quality, lineage, validation, monitoring, and client-facing products.