Top Senior Data Engineer Jobs in Atlanta, GA

Own the architecture, development, and operations of Cargill’s enterprise data platform. Build and optimize Azure and on-premises ETL/ELT pipelines, manage databases and migrations, implement CI/CD, improve performance and reliability, and create embedded Superset dashboards. Monitor platform health, data quality, security, and costs while integrating emerging AI capabilities. Mentor data professionals and make architectural decisions across global teams.

Designs, builds, and maintains scalable data infrastructure, streaming and batch pipelines, data systems, data models, and cloud-based data products. Develops automated deployment pipelines, evaluates architectures, supports data governance, and collaborates with analytics teams and stakeholders. The role requires advanced programming and SQL expertise, modern data architecture experience, and proficiency with cloud platforms, Spark, Kafka, orchestration tools, and data modeling frameworks.

Designs, implements, and maintains secure, scalable ETL pipelines, data warehouses, and Azure data environments. Responsibilities include developing SSIS and Azure Data Factory solutions, creating Azure DevOps CI/CD pipelines, testing and debugging code, documenting procedures, improving data infrastructure, managing projects, and supporting SharePoint sites. The role requires collaboration with stakeholders, strong SQL and database design skills, and independent problem-solving.

Bridges procurement stakeholders, governance teams, business analysts, and engineers to define data contracts, map procurement data to OCDS, and build Databricks pipelines. Responsibilities include requirements gathering, source-to-target mapping, data modeling, validation, quality controls, lineage, governance, and production support. Develops Bronze, Silver, and Gold data layers using Spark, PySpark, Python, SQL, Delta Lake, and Databricks services while supporting Agile delivery, testing, deployment, and troubleshooting.

The AVP, Data Engineer will design and build data ingestion, transformation, and replication pipelines using SQL, Python, dbt, and Dagster. Responsibilities include translating requirements into technical designs, developing data products, debugging production issues, supporting testing, and improving data engineering practices. The role also involves data modeling, mentoring junior engineers, conducting code reviews, and providing technical leadership while remaining hands-on. Experience with modern data platforms, workflow orchestration, cloud ecosystems, and insurance or financial services is preferred.

Build and operate production data pipelines connecting marketing, CRM, licensing, payment, and operational systems with Databricks. Develop PySpark and SQL ETL/ELT workflows, dimensional models, APIs, data quality monitoring, identity resolution, and customer segmentation data flows. Support HubSpot enrichment, lifecycle marketing, campaign attribution, funnel reporting, and marketing ROI analytics while maintaining documentation, alerting, and incident response processes.

Designs and maintains production data pipelines, models claims and operational data, and improves reliability through testing, monitoring, and alerting. Owns workflow troubleshooting, backfills, performance optimization, documentation, access controls, and CI/CD practices. Partners with business and technical teams to define requirements and deliver trusted datasets using SQL, Python, cloud data warehouses, and orchestration tools.

Designs trusted Snowflake data products and enterprise semantic models that standardize business concepts, metrics, relationships, and rules. Builds Cortex semantic views and agents, validates AI-generated results, develops regression tests, and maintains metadata, lineage, and documentation. Leads modeling initiatives from discovery through production while partnering with business and technical stakeholders to improve analytics, automation, and AI consistency.

Build and maintain production data ingestion, transformation, orchestration, and modeling pipelines using Dagster or Airflow, dbt, Snowflake, Python, Airbyte, and AWS. Monitor and debug production workflows, manage backfills and incremental loads, configure infrastructure, review code, and document runbooks. Partner with analytics and product teams on dimensional modeling, data quality, and pipeline reliability.

Designs, builds, and operates large-scale Hadoop-based data platforms and pipelines. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark applications; managing streaming, workflow orchestration, and distributed data systems; troubleshooting production environments; and supporting analytics, machine learning, and reporting workloads. The role also involves cloud-based Hadoop operations, lakehouse technologies, governance tooling, Kubernetes platforms, CI/CD, and infrastructure-as-code practices.

Designs and delivers scalable data pipelines, models, repositories, dashboards, and reporting solutions. Partners with business and technology stakeholders to translate requirements into technical solutions, supports ETL/ELT processes, ensures data quality and performance, leads projects, mentors team members, and promotes automation and continuous improvement across Guardian Pharmacy’s data platform.

Build and maintain scalable batch, streaming, and real-time data pipelines, platforms, data lakes, warehouses, and lakehouses. Implement data quality, governance, security, observability, automation, and cloud modernization. Partner with product, engineering, analytics, data science, and business teams to deliver analytics, AI/ML, reporting, and customer insight solutions. Troubleshoot production issues, guide architecture, mentor engineers, and provide technical leadership across marketing data initiatives.

Designs enterprise streaming and Kappa Architecture frameworks that enable scalable, event-driven data processing in Microsoft Fabric. Responsibilities include creating replay and backfill mechanisms, reusable stateful streaming templates, idempotent processing patterns, schema-drift handling, Kafka and Event Hubs ingestion, stream-health dashboards, and latency controls. The role serves as the internal authority on production streaming architecture and communicates complex real-time concepts to technical and executive stakeholders.

Lead the redesign and implementation of a scalable Azure and Databricks lakehouse. Responsibilities include migrating Azure Data Factory pipelines, building Delta Lake architecture, developing SQL and Python transformations, establishing engineering standards, monitoring data quality, partnering with stakeholders, leading technical projects, mentoring engineers, and supporting future team growth.

Lead the design and delivery of scalable batch and real-time data pipelines across AWS, Snowflake, PostgreSQL, MongoDB, dbt, and Kafka. Establish standards for data modeling, governance, security, testing, observability, and reliability. Build trusted datasets and semantic layers for BI, analytics, AI/ML, RAG, and LLM applications. Drive platform optimization, data integration, MDM, technical strategy, mentoring, and cross-functional collaboration.

Designs and maintains scalable, low-latency data pipelines for real-time and batch market data. Builds ETL/ELT processes, optimizes storage and streaming systems, supports productionizing AI and machine-learning models, and improves data quality, observability, reliability, and cost efficiency. Collaborates with data scientists, ML engineers, and product teams to evolve Fetcherr’s data platform.

Lead the architecture, development, governance, and operation of scalable data platforms, pipelines, schemas, databases, and warehousing infrastructure. Guide data acquisition, transformation, cataloging, and integration across cloud and local environments while ensuring reliability, security, performance, and data quality. Collaborate with product and engineering leadership, recommend data solutions, establish technical standards, and mentor engineering teams.

Design, build, and optimize modern Azure and Databricks data platforms for clients. Develop batch and streaming ingestion pipelines, lakehouse and medallion architectures, dimensional models, and transformation workflows. Lead discovery sessions, requirements workshops, architecture discussions, and solution reviews. Recommend practices for governance, lineage, monitoring, DevOps, security, and data modeling while translating business needs into scalable technical solutions and communicating with technical and executive stakeholders.

Lead design, build, and maintain scalable data pipelines, ETL processes, data models, and storage solutions. Support production data ecosystem, optimize performance, ensure data quality and security, integrate data sources/APIs, and provide technical guidance to teams while collaborating with stakeholders.

Leads data engineering for Payment Operations monitoring transformation. Designs and maintains Snowflake pipelines, tables, stored procedures, monitoring scenarios, KPI infrastructure, and data quality controls. Serves as the Snowflake subject matter expert, documents architecture and operational processes, guides technical partners, and translates business needs into scalable data solutions. The role requires production data engineering experience, advanced SQL, Snowflake expertise, job scheduling, pipeline operations, and stakeholder communication.

Designs and leads scalable real-time and batch data pipelines, data platforms, architectures, data products, and intelligent APIs. Ensures data quality, supports BI and AI/ML initiatives, and develops ingestion, processing, migration, and analytical solutions for big data, NoSQL, and data warehouse environments. Provides technical leadership, mentorship, code reviews, testing, CI/CD implementation, system monitoring, troubleshooting, and cross-functional translation of business requirements into reliable cloud-based solutions.

Design and build cloud-native, scalable data solutions for systematic trading. Partner with traders, quants, and architects to develop Snowflake-based ETL, data models, and pipelines; lead AI-driven capability adoption; ensure security and SDLC best practices.

Design, implement, and maintain cloud-based data pipelines and warehousing/lake solutions. Advise clients on best practices, lead technical decisions, mentor engineers, and collaborate with stakeholders to deliver scalable, maintainable data engineering solutions.

As a Lead Data Engineer, you will architect data foundations for AI applications, lead a team, optimize data pipelines, and ensure data governance.

Design and implement AI-optimized data platforms and production ETL/ELT pipelines, build semantic and analytical layers, write production-ready SQL/Python code, collaborate with stakeholders and analytics teams, and contribute to the firms data engineering practice and client delivery.