Top Data Engineer Jobs in Minneapolis, MN

Lead performance investigations across the data product stack to diagnose and fix slow queries and UX issues. Design data architecture changes, build benchmarks, measure latency/throughput/cost (ClickHouse Cloud), prototype solutions, instrument monitoring, and partner with product and engineering to balance responsiveness, scalability, and cost.

Design, build, and operate large-scale data pipelines and ETL processes to normalize varied partner datasets. Lead technical decisions, mentor engineers, implement testing and monitoring, manage data warehouses/lakes, and collaborate cross-functionally to deliver robust data infrastructure that supports ML and product teams.

Define and lead the technical architecture for a lakehouse on AWS, build production pipelines and data models for time-series and multi-source data, introduce data governance and catalog foundations, author platform playbooks (ADRs, runbooks, Terraform modules), deliver projects end-to-end, and mentor engineers while representing the platform to senior leadership and non-technical stakeholders.

Founding data engineer to design and build Payabli's data platform: architect lakehouse/warehouse, build batch and streaming pipelines, model canonical datasets, ensure data quality/observability, enforce access/masking/lineage for regulated financial data, and enable analytics/ML feature pipelines while establishing team standards and CI/CD.

The Senior Data Engineer will design and implement data processing pipelines using Java and Spark, optimize performance, and mentor junior engineers while ensuring compliance with data governance standards.

Lead design and delivery of scalable production data pipelines and data products (Spark/Python/SQL) on Databricks; integrate diverse federal health systems, define reusable ingestion patterns, ensure data quality/governance, mentor engineers, and troubleshoot/optimize production workflows.

Design, build, and operate scalable batch and streaming data pipelines and a lakehouse on AWS. Implement ETL/ELT, CDC, metadata/cataloging, lineage, governance, data quality, IaC, and CI/CD for reliable, observable, and secure data platforms to support analytics and ML.

Build and maintain scalable ETL/data pipelines and internal frameworks to support ML and product teams. Drive architecture, observability, data quality, and engineering best practices across AI/ML workflows and prompt-engineering pipelines.

Lead design, build, and operate scalable Databricks/Delta Lake data pipelines and Kimball dimensional models using dbt. Manage governance with Unity Catalog, optimize performance and cost, operationalize ML with MLflow, and coordinate/quality-check offshore vendor engineering. Mentor engineers, uphold standards, and deliver trusted data products that power analytics, reporting, and MLOps workflows.

Lead SAP S/4HANA data migration activities: extract, map, transform, load, validate, and reconcile master and transactional data using BODS, Migration Cockpit, LSMW/BDC and HANA SQL. Support mock loads, cutover execution, defect remediation, and collaborate with functional owners to cleanse and correct source data.

Design, build, and maintain scalable data pipelines, data models, ETL/ELT, and BI solutions. Collaborate with data scientists and stakeholders to improve data quality, accessibility, and PHI security. Develop and document workflows, optimize processing/storage, support data warehouse and visualization, and adopt AI tools to drive data-informed outcomes.

The Senior Data Engineer will construct and optimize data pipelines, work with cross-functional teams, ensure data quality and governance, and mentor junior engineers while leading advanced analytics initiatives.

Design, build, and operate scalable data transformation pipelines and dimensional models using dbt, SQL, and AWS. Implement infrastructure-as-code (AWS CDK), CI/CD, and query performance optimizations while ensuring data quality, governance, and collaboration with cross-functional and offshore teams.

Lead Data Engineer builds and maintains scalable data pipelines and lakehouse/warehouse platforms (Databricks) to support analytics, BI, reporting, and AI. Responsibilities include data modeling, ETL/ELT, medallion architecture, orchestration and monitoring (Airflow/Cron), performance tuning, CDC/incremental loads, data quality, and leading/mentoring a team while collaborating with stakeholders and maintaining documentation.

Design and build cloud-based data applications and pipelines to analyze clinical data, enrich and provision datasets, and support clinical/operational processes. Mentor junior engineers, drive R&D for repeatable templates, collaborate with Product/Platform/Architecture, and prioritize source control, documentation, and simple solutions using modern data, ETL, and LLM tooling.

Architect, build, and maintain Snowflake-based ELT pipelines and production data models. Implement observability, data quality, and governance; optimize cost and freshness. Partner with Analytics, Product, and Engineering, use dbt/CI-CD and AI-assisted tooling, and participate in reviews and architecture planning to support analytics, BI, and ML.

Design, build, and maintain scalable ETL/ELT pipelines into Snowflake, enforce data quality, optimize data models and query performance for Power BI reporting, produce documentation, and troubleshoot data issues while collaborating with stakeholders.

Design, implement, and maintain PySpark-based data reconciliation solutions for financial systems. Build matching algorithms, integrate with rules engines, process large distributed datasets, identify and resolve discrepancies, and collaborate with analysts and architects to improve data quality and governance.

Design, implement, and maintain data quality processes using cloud platforms (Azure/AWS/GCP). Collaborate with stakeholders to define standards, perform data profiling and cleansing, document issues and remediation, generate quality metrics, and support cross-functional teams to improve data architectures and integration.

Design, implement, and maintain PySpark applications to automate large-scale financial data reconciliations. Build transformation and matching algorithms, integrate with rules engines, analyze data gaps, and collaborate with analysts and architects to ensure data quality and system resilience.

Design, build, and optimize PySpark applications and ETL pipelines to process large-scale datasets from SQL/NoSQL sources, data lakes, and streaming platforms. Ensure data quality, error handling, performance tuning, and collaborate with analysts, scientists, and architects to deliver scalable data solutions.

Design, develop, test, and deploy high-performance Spark/Scala data processing applications and ETL pipelines using the Cloudera Hadoop ecosystem. Optimize Spark and platform performance, ensure data integrity and security, collaborate with data scientists and analysts, troubleshoot issues, and implement version control and CI/CD for Spark applications.

Design and implement data federation and lakehouse architectures, build scalable ETL/ELT pipelines with Python and Spark, optimize performance across federated queries, manage Delta/Iceberg/Hudi tables, and enforce governance, security, and access controls for analytics and AI teams.

Design and develop Big Data applications using Java, Spark, and MapReduce on Cloudera/Hadoop ecosystems. Work with Hive, Impala, YARN, Kafka, and large datasets; write tests (JUnit), perform data analysis, and script in Unix/Python. Lead and manage global technology teams and follow industry best practices.

Design and implement big data solutions (Spark, Hive, Java, CDP). Analyze and consolidate disparate data sources, produce functional specifications, review data models, gather stakeholder requirements, validate implementations, support production deployments, investigate data quality and data lineage, and collaborate with technology leads to ensure data completeness and accuracy.