Top Data Engineer Jobs in Madison, WI

Designs, develops, and supports analytics data solutions for clients, including Snowflake and dbt ingestion and transformation pipelines, data marts, validation, monitoring, governance, and documentation. Manages multiple consulting projects, budgets, timelines, quality assurance, troubleshooting, client communication, and implementation planning. Requires strong data engineering experience, regulated-industry knowledge, stakeholder communication, and willingness to travel.

Administer and enhance Edmunds’ Databricks data platform and AWS infrastructure. Build and maintain ETL pipelines, infrastructure-as-code tooling using Terraform or CDK, and operational dashboards, alerts, and reports. Collaborate with business, engineering, analytics, security, and infrastructure teams to support data platform users and AI solutions. Evaluate new technologies, troubleshoot platform issues, and improve operational, cost, and security visibility.

Define and govern enterprise-scale data architecture across batch, streaming, warehouse, lakehouse, transactional, and AI use cases. Establish standards for data quality, lineage, access, cataloging, governance, observability, and SLAs. Architect AI-enabled workflows, resolve complex architecture issues, influence roadmaps, and mentor engineers through hands-on technical leadership. The role requires 15+ years of software, data engineering, or architecture experience and expertise in large-scale data platforms and modeling.

Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.

Design, build, and maintain scalable Spark-based ETL pipelines and computed tables in a central data lake. Integrate structured and unstructured IoT, sensor, and external data for analytics, model training, and dashboards. Collaborate with Data Science, Analytics, and ML teams to ensure reliable, high-quality customer-facing datasets.

Build and support scalable data platforms and pipelines, leading migrations such as Snowflake to BigQuery and transitioning reporting to Looker. Responsibilities include data architecture, ETL/ELT, API and marketing integrations, data quality, production troubleshooting, warehousing, performance optimization, and platform modernization. The role partners with analytics and business teams, owns projects through production, documents solutions, and provides technical guidance.

Configure and maintain Adobe Experience Platform and Real-Time CDP data pipelines, XDM schemas, datasets, identity resolution, Profile enablement, segmentation readiness, and destination activation. Validate data quality, troubleshoot ingestion and identity issues, manage sandboxes, and implement privacy, consent, governance, and access controls. Partner with architects, data engineers, consultants, analysts, data scientists, and marketing teams to support reporting, personalization, and machine learning readiness.

Build scalable, secure data pipelines, models, platforms, and governance systems that enable AI applications to access trusted enterprise data. Partner with analytics, marketing, and product teams to translate requirements into technical solutions, independently deliver projects through production, and support sensitive-data security and access controls. Preferred experience includes LLM architectures, cloud data platforms, infrastructure as code, regulated environments, and technical leadership.

Design and scale lakehouse architecture, including Bronze, Silver, and Gold data layers. Build reliable streaming and batch pipelines with Kafka, Spark, and Airflow; manage Iceberg, Delta, and Hudi table formats; optimize Trino, Starburst, and Databricks queries; and improve platform reliability, data quality, and performance. Collaborate with data scientists, analysts, and product teams to deliver scalable data platform solutions.

Design and scale lakehouse architecture, streaming and batch pipelines, and reliable data platforms using Kafka, Spark, Airflow, Iceberg, Databricks, and related technologies. Build Medallion-layer data systems, manage open table formats, optimize distributed queries, monitor platform reliability, resolve data quality issues, and collaborate with analysts, data scientists, and product teams.

Build and maintain scalable ETL/ELT pipelines and data foundations using Databricks, dbt, and AWS. Transform raw structured and unstructured data into reliable datasets for product analytics and data science, including telemetry, streaming data, and feature engineering. Develop dashboards in Tableau and Databricks, optimize workflows, and collaborate with Product, Engineering, analytics, and data science stakeholders to support product health, adoption, and performance insights.

Design, build, and maintain scalable batch and streaming data pipelines, cloud infrastructure, and the Enterprise Data Hub. Partner with stakeholders to productionize ingestions, ensure data quality, implement metadata/catalog solutions, mentor junior engineers, and support analytics and ML workloads using cloud and big-data tooling.

Build and maintain large-scale data processing pipelines using Apache Spark (PySpark/Scala) and Python. Design ETL workflows, implement distributed data processing, use version control (Git), and work with cloud platforms (GCP/AWS/Azure).

Design and optimize Snowflake data warehouses, dbt transformation models, and scalable ETL/ELT pipelines. Develop ingestion utilities, improve SQL and warehouse performance, establish data quality and governance controls, and maintain CI/CD workflows using Git. Collaborate with analysts, data scientists, and business stakeholders across manufacturing, supply chain, and operations. The role also supports modern DataOps practices and may involve orchestration tools and cloud platforms.

Build and maintain scalable ETL/ELT pipelines using Azure Data Factory, Snowflake, dbt, and SQL. Develop data models, transformations, tests, and documentation; ingest data from APIs, databases, files, and enterprise applications; optimize performance; ensure data quality; troubleshoot failures; and support production deployments and monitoring.

Design, build, and maintain Keboola-based data pipelines integrating multiple business systems to deliver accurate, scalable data for supply chain, logistics, and transportation analytics; collaborate with analysts and engineers; support documentation, governance, and pipeline automation.

Designs, builds, tests, maintains, and optimizes large-scale data pipelines, ETL processes, ingestion workflows, and cloud data storage. Develops Databricks data engineering and AI/ML pipelines, Azure Data Lake solutions, Power BI dashboards, and CI/CD automation. Ensures data quality, security, scalability, cost efficiency, and reusable engineering practices while collaborating with distributed technical teams.

Build, model, and maintain scalable BigQuery-based data solutions on GCP. Implement performant data models, storage partitioning/clustering, ETL improvements, data validation, and documentation. Collaborate with architects, data scientists, and engineers to deliver governed, high-quality data for reporting and AI initiatives.

Designs, builds, deploys, and maintains scalable data architectures, databases, ETL/ELT pipelines, API integrations, and cloud data solutions in AWS GovCloud. Migrates on-premises databases to cloud platforms, optimizes data models and performance, implements governance and security controls, and supports reporting and analytics. Collaborates with architects, data scientists, and stakeholders while documenting systems and conducting integration, validation, cost, and risk assessments.

Designs, builds, and maintains cloud data pipelines and ETL/ELT processes supporting VA healthcare analytics. Responsibilities include patient matching, record linkage, data harmonization, quality management, documentation, performance optimization, and pipeline support. Collaborates with analysts and stakeholders to develop scalable solutions and Power BI reports using Databricks SparkSQL. The role requires federal healthcare data experience, strong SQL or Spark SQL skills, and the ability to obtain a VA Public Trust clearance.

Entry-level data engineering development program involving three rotations, cloud-native data platforms, batch and real-time pipelines, distributed systems, ETL/ELT, data modeling, AI platforms, and enterprise data products. Participants use AWS, Spark, Java, Python, SQL, Snowflake, Kafka, and related technologies while gaining experience with LLM architectures, APIs, microservices, and secure production deployments.

Data Engineer Interns will gain hands-on experience building batch and real-time data pipelines using AWS, Spark, Java, Python, and SQL. They will learn data modeling, ETL/ELT, distributed systems, streaming technologies, and AI platforms, including LLM, RAG, vector database, and agentic AI architectures. Interns will collaborate with data engineers, contribute to secure data and AI solutions, and participate in mentorship and professional development activities.

Supports the design and development of scalable end-to-end data pipelines and cloud data platforms. Responsibilities include improving data reliability and quality, implementing data management practices, supporting automation and process improvements, ensuring secure and cost-effective solutions, and recommending system enhancements. The intern collaborates with global teams and gains hands-on experience in data engineering, project coordination, and innovation management.

The Principal Data Engineer designs and builds scalable cloud-native data platforms, pipelines, frameworks, and governed data products supporting analytics, reporting, AI, and operational workloads. Responsibilities include Databricks, Spark, Delta Lake, Azure, SQL Server, batch and streaming processing, data integration, performance optimization, governance, modernization, troubleshooting, architecture reviews, and mentorship. The role provides senior technical leadership without direct people management and supports insurance-domain data across policy, claims, billing, finance, and regulatory functions.

Design, build, and maintain data pipelines and a Microsoft Fabric data lakehouse to ingest, transform, and secure large datasets. Collaborate with analysts and Power BI developers, ensure data quality, troubleshoot issues, and optimize cloud-based ETL/ELT and warehousing solutions.