Top Senior Data Engineer Jobs in Portland, ME

Design and scale Zip’s enterprise data and AI platform across batch, streaming, structured, and unstructured workloads. Build reliable pipelines, AI-ready data products, semantic models, vector and hybrid search, and RAG capabilities. Establish standards for modeling, testing, governance, security, observability, and automation. Partner with engineering, data science, product, and business teams while mentoring engineers and influencing platform architecture.

Design and build modern GCP lakehouse architectures, including Python and Spark pipelines, BigQuery ingestion, Kafka CDC, Delta Lake and Iceberg UniForm integrations, Delta Sharing endpoints, Snowflake and Databricks connectivity, data lineage, governed access controls, and semantic-layer alignment. Collaborate cross-functionally, troubleshoot full-stack issues, and deliver reusable data-sharing adapters and end-to-end data features.

Leads enterprise data engineering initiatives by designing and optimizing data pipelines, data lakes, warehouses, and analytics infrastructure. Uses Snowflake, Informatica, SQL, and related technologies to integrate diverse data sources, improve data quality, secure PII, and support analytics and data science teams. Establishes governance, monitoring, scalability, performance, and cost-efficiency practices while mentoring teammates and partnering with stakeholders on complex technical solutions.

Leads data and analytics engineering for Zero Trust maturity assessment and implementation measurement. Designs common data models, integrates and validates enterprise data, develops maturity and risk metrics, supports prioritization and gap analysis, and establishes data quality, lineage, traceability, and documentation practices. Partners with dashboard and automation specialists to streamline reporting and data refresh workflows.

Own and operate the Snowflake warehouse and Sigma semantic layer; manage ingestion (Fivetran), dbt model scoping and review, reverse-ETL, data quality SLAs, dashboards, month-end financial analytics, vendor management, and automation to support AdOps, Sales, Finance, and executive reporting.

Design, build, and maintain scalable ETL/ELT data pipelines and data platform architecture. Build programmatic pipelines (primarily Python), optimize analytical data models (SQL), integrate sources into Snowflake, use orchestration/transformation tools (dbt, Airflow, Dagster), improve data quality/observability, and partner with product, engineering, and analytics to deliver reliable data for decision-making.

Design and maintain a dbt-based business data layer for GTM metrics, centralize and reconcile multi-source data, implement automated data quality and reconciliation checks, align and document metrics with stakeholders, mentor analytics team, and lead root-cause investigations into complex data anomalies.

Designs and operates scalable serverless AWS data pipelines, data models, ETL/ELT workflows, feature stores, and analytics-ready data assets. Partners with AI, Analytics, Product, and Underwriting teams to support reproducible ML training and scoring, MLOps, data quality, monitoring, governance, and production troubleshooting. The role handles sensitive healthcare and financial data, optimizes serverless workloads, supports batch and near-real-time processing, and applies infrastructure-as-code, CI/CD, documentation, and responsible data-use practices.

Design, build, and maintain scalable data pipelines and integrations across AWS, Azure, and BigQuery. Normalize, aggregate, and ensure data quality for large web and transactional datasets, leveraging APIs, ETL/orchestration tools, and big-data technologies to support analytics and BI in an e-commerce environment.

Lead performance investigations across the data product stack to diagnose and fix slow queries and UX issues. Design data architecture changes, build benchmarks, measure latency/throughput/cost (ClickHouse Cloud), prototype solutions, instrument monitoring, and partner with product and engineering to balance responsiveness, scalability, and cost.

Design, build, and operate production-grade near-real-time data pipelines and APIs on AWS (Kinesis, Glue/Spark, Lambda, EMR). Implement IaC and CI/CD, optimize data lakes and operational models, improve observability and reliability, and participate in on-call support.

Founding data engineer to design and build Payabli's data platform: architect lakehouse/warehouse, build batch and streaming pipelines, model canonical datasets, ensure data quality/observability, enforce access/masking/lineage for regulated financial data, and enable analytics/ML feature pipelines while establishing team standards and CI/CD.

The Senior Data Engineer will design and implement data processing pipelines using Java and Spark, optimize performance, and mentor junior engineers while ensuring compliance with data governance standards.

Lead design and delivery of scalable production data pipelines and data products (Spark/Python/SQL) on Databricks; integrate diverse federal health systems, define reusable ingestion patterns, ensure data quality/governance, mentor engineers, and troubleshoot/optimize production workflows.

Design, build, and operate scalable batch and streaming data pipelines and a lakehouse on AWS. Implement ETL/ELT, CDC, metadata/cataloging, lineage, governance, data quality, IaC, and CI/CD for reliable, observable, and secure data platforms to support analytics and ML.

Build and maintain scalable ETL/data pipelines and internal frameworks to support ML and product teams. Drive architecture, observability, data quality, and engineering best practices across AI/ML workflows and prompt-engineering pipelines.

Lead design, build, and operate scalable Databricks/Delta Lake data pipelines and Kimball dimensional models using dbt. Manage governance with Unity Catalog, optimize performance and cost, operationalize ML with MLflow, and coordinate/quality-check offshore vendor engineering. Mentor engineers, uphold standards, and deliver trusted data products that power analytics, reporting, and MLOps workflows.

Design, build, and operate scalable data transformation pipelines and dimensional models using dbt, SQL, and AWS. Implement infrastructure-as-code (AWS CDK), CI/CD, and query performance optimizations while ensuring data quality, governance, and collaboration with cross-functional and offshore teams.

Lead Data Engineer builds and maintains scalable data pipelines and lakehouse/warehouse platforms (Databricks) to support analytics, BI, reporting, and AI. Responsibilities include data modeling, ETL/ELT, medallion architecture, orchestration and monitoring (Airflow/Cron), performance tuning, CDC/incremental loads, data quality, and leading/mentoring a team while collaborating with stakeholders and maintaining documentation.

Architect, build, and maintain Snowflake-based ELT pipelines and production data models. Implement observability, data quality, and governance; optimize cost and freshness. Partner with Analytics, Product, and Engineering, use dbt/CI-CD and AI-assisted tooling, and participate in reviews and architecture planning to support analytics, BI, and ML.

Design and implement data federation and lakehouse architectures, build scalable ETL/ELT pipelines with Python and Spark, optimize performance across federated queries, manage Delta/Iceberg/Hudi tables, and enforce governance, security, and access controls for analytics and AI teams.

Design, build, test, and maintain high-performance Python data applications and backend services. Lead technical design, review code, mentor junior engineers, integrate with databases and cloud services, optimize performance, and ensure security, scalability, and reliability.

Design, develop, maintain, and optimize enterprise Tableau dashboards and BI solutions. Integrate and cleanse data from Oracle/Sybase/Big Data, tune SQL, automate Tableau extracts, manage Tableau Server on Linux, support testing, UAT, deployments, and troubleshoot data integrity and performance issues.

Design, build, and maintain scalable batch and streaming data pipelines, improve data modeling and governance, contribute to enterprise data architecture, and collaborate with cross-functional teams to ensure reliable, high-quality data for analytics and operational use.

Design, build, and maintain an internal data platform enabling secure, governed access for engineering and analytics. Partner with architects, implement data pipelines and automations, enforce security and compliance, support platform services, recommend architecture improvements, mentor teams, and drive adoption of scalable, self-service data capabilities.