Top Senior Data Engineer Jobs in Portland, ME

Develop and maintain large-scale, cloud-native data services and distributed pipelines for multi-tenant healthcare systems. Responsibilities include data modeling, database component development, SQL programming, performance tuning, testing, deployment, code reviews, proof-of-concepts, and full software development lifecycle ownership. Collaborate with global engineering teams, evaluate technologies, apply security and operational requirements, and use AI tools to accelerate development.

Leads the design, automation, optimization, and maintenance of scalable data pipelines using Airflow, dbt, Python, SQL, and Google Cloud technologies. Manages dbt models, develops reusable architecture patterns, monitors pipeline performance, supports CI/CD and testing, and ensures data security and governance. The role also provides technical leadership, mentors engineers, collaborates cross-functionally, documents implementations, and drives continuous improvement across data engineering practices.

Build and maintain scalable batch and real-time data pipelines, model data to power analytics and ML, and operate distributed data systems. Collaborate with business, engineering, and data science teams to ensure data quality, documentation, and debugging while adopting new technologies and working in an ambiguous, fast-changing environment.

Build and maintain low-latency, highly available distributed data systems and services to deliver high-quality data for product, analytics, and ML. Partner with data scientists and product teams to design scalable, observable, multi-tenant data products using batch and real-time processing frameworks. Ensure 24x7 reliability, monitoring, and operational excellence while owning end-to-end delivery.

Design, build, and own large-scale batch and streaming data pipelines and core Ads data products. Collaborate with analytics, data science, and engineering teams to model, transform, govern, and deliver high-quality ad datasets for analysis, ML, and reporting while ensuring privacy compliance.

Build and own production data pipelines using Snowflake, Airflow, dbt, and Python; implement canonical data models; embed validation, quality checks, lineage, freshness, and alerting; support governed self-service data delivery; migrate workloads; partner with business consumers; participate in on-call and incident reviews; and mentor engineers while maintaining strong testing, CI/CD, documentation, and code-review practices.

Lead database engineering for a federal species-data integration program in a hybrid AWS cloud environment. Responsibilities include harmonizing schemas, documenting source-to-target mappings and lineage, enforcing metadata and data-quality standards, implementing encryption and NIST-aligned security controls, building authoritative source registries, optimizing performance, and leading production migration, validation, monitoring, and operational documentation. The role requires Databricks or equivalent lakehouse expertise, ETL programming, federal governance knowledge, and database migration leadership.

Designs secure Azure data pipelines and analytics foundations using Databricks and ADLS Gen2. Responsibilities include modernizing Oracle workloads, developing medallion data models, implementing data quality and monitoring practices, integrating curated datasets with Power BI, and establishing secure Azure Government identity, network, storage, and access patterns. The role also requires documentation, operational support, stakeholder collaboration, and performance optimization.

Designs, develops, and sustains cloud-based data analytics infrastructure and ETL pipelines. Manages data retention, curation, cataloging, schemas, and automation across AWS, Elasticsearch, and Google Analytics platforms. Creates visualizations and reports using Kibana, AWS QuickSight, and Google Analytics 4. Supports cloud-native architecture, infrastructure automation, DevOps, monitoring, troubleshooting, bug fixes, and customer support.

Leads data engineering strategy and mentors an analytics engineering team. Designs scalable ELT pipelines, warehouse integrations, data models, API ingestion workflows, and data quality frameworks using cloud and database technologies. Conducts exploratory analysis, supports dashboards and analytical solutions, monitors warehouse performance and costs, and ensures data governance, security, and compliance. Collaborates with data scientists, analysts, engineers, and business stakeholders to deliver healthcare analytics solutions.

Own SQL and dbt-based data modeling and data quality for clinical and claims datasets. Author and refactor dbt models/tests, triage DQ alerts, design monitors, optimize SQL performance, and collaborate with Data Engineering and stakeholders to ensure trusted production datasets.

The job involves building responsive web dashboards, advanced data visualizations, and analytics tools for unmanned aircraft systems, requiring a deep understanding of UI/UX design and data presentation.

Lead the data engineering strategy and architecture for distributed pharmaceutical data systems. Design and scale pipelines, APIs, and data platforms while ensuring reliability, governance, quality, and PII/PHI protection. Drive technical decisions, improve developer efficiency, deliver business impact, influence cross-functional roadmaps, mentor engineers, communicate with executives, and promote organizational adoption of new technologies and processes.

Lead financial data engineering initiatives involving commissions, incentive programs, and backend financial systems. Build scalable, accurate, and reliable data pipelines, platform tooling, partner integrations, analytics infrastructure, and data science infrastructure. Shape technical strategy and roadmaps, collaborate with Finance, Product, Data Science, and Engineering, mentor engineers, contribute code, lead incident response, and establish engineering standards.

Design and operate Alpaca’s scalable data platform on GCP, including lakehouse infrastructure, Kubernetes-based deployments, streaming and CDC pipelines, batch ingestion, BI access, cataloging, monitoring, and governance. Build infrastructure as code with Terraform and Ansible, operate distributed query engines and Apache Iceberg, maintain reliability practices, and collaborate with DevOps and analytics teams to support rapidly evolving data requirements.

Develop business intelligence and analytics solutions using financial, operational, RF, and video performance data. Build scalable data models, dashboards, reports, automation workflows, and ETL/ELT processes using Power BI, Tableau, DataMiner, and Qligent. Analyze trends and anomalies, monitor satellite and video delivery performance, translate business requirements into actionable insights, and collaborate with enterprise stakeholders in a fully remote consulting role.

Designs and maintains scalable enterprise data pipelines, models, and data products supporting analytics, reporting, Power BI, AI agents, and machine learning. The role implements layered data architectures, ensures data quality and performance, integrates data across platforms, troubleshoots pipeline issues, and collaborates with architects, analysts, and product owners. AI-assisted development tools such as GitHub Copilot and Claude are used to build AI-ready data solutions.

Design, build, and operate scalable cloud-native data platforms and pipelines supporting analytics, reporting, AI, and customer-facing applications. Develop ETL/ELT workflows, data lakes, event-driven streaming solutions, integrations, and data models using Python, SQL, AWS, databases, warehouses, messaging technologies, and orchestration tools. Monitor production systems, optimize performance, automate testing and deployment, and collaborate with engineering, platform, product, and data teams.

Design, build, and operate scalable cloud-native data platforms and pipelines supporting analytics, reporting, AI workloads, and customer-facing applications. Develop ETL/ELT processes, data lakes, event-driven and streaming solutions, data models, enterprise integrations, and automated monitoring and deployment systems. Collaborate with software, platform, product, and data teams to ensure reliable, secure, high-performance production systems.

Leads the design and development of Mozilla’s data platform, processing tens of terabytes daily. Builds GCP workflows using Dataflow, BigQuery, Pub/Sub, GKE, and Airflow; evaluates technologies; establishes engineering standards; partners with Data Science, Product, Legal, Policy, and Engineering; develops data abstractions; mentors engineers; and supports evolving data products.

Design and improve large-scale data processing and platform systems handling tens of terabytes daily. Build GCP workflows using Dataflow, BigQuery, Pub/Sub, GKE, and Airflow; evaluate technologies; establish engineering standards; lead cross-functional projects; mentor engineers; collaborate with Data Science; and develop data abstractions and active data products.

Designs and builds production data systems, pipelines, database schemas, analytical models, and internal platforms. Responsibilities include writing tested code, optimizing queries and storage, controlling infrastructure costs, enforcing CI/CD, and implementing data testing, monitoring, and alerting. The role also develops AI and LLM-backed data use cases, including retrieval pipelines, embeddings, vector storage, evaluation harnesses, and agentic workflows.

Build and maintain ETL and data pipelines using Python, PySpark, and AWS services. Orchestrate workflows, develop event-driven integrations, optimize data storage and queries, process API and JSON data, and implement data quality monitoring. Support CI/CD, production operations, cloud migration, data lake and lakehouse initiatives, and near-real-time systems while collaborating with stakeholders to deliver technical solutions.

Builds, operates, and owns enterprise data warehouse pipelines and models on Databricks. Develops Delta Lake batch and streaming pipelines across Bronze, Silver, and Gold layers; implements data quality, reconciliation, observability, governance, and performance improvements. The role owns assigned data domains, supports event-driven ingestion, documents architecture and runbooks, responds to incidents, collaborates with QA and governance teams, and mentors data engineers.

As a Data Engineer, you will design and maintain data pipelines, support DoD systems, and ensure data quality and compliance.