Top Data Engineer Jobs

Build scalable data ingestion, transformation, entity resolution, and integration pipelines for a cybersecurity ontology and knowledge graph platform. Design batch and real-time distributed systems processing billions of security events, ensure data quality and governance, model entities and relationships, and enable AI agents through structured security knowledge and ontology APIs.

The Senior Data Engineer will design, implement, and manage data pipelines for real-time analytics and ML training datasets, ensuring efficient data flow and quality in a hybrid environment.

Designs, builds, deploys, and operates scalable data pipelines and cloud-based data architectures supporting lending analytics across Mortgage and Personal Loans. Responsibilities include data ingestion and transformation, schema design, SQL optimization, workflow orchestration, monitoring, troubleshooting, data quality, and generative AI integration. The role also establishes engineering standards and collaborates with analysts, engineers, and governance teams to deliver reliable production data systems.

Build and maintain Python-focused data analytics applications, pipelines, integrations, batch workflows, and data-quality solutions for Prime Services financial data. Partner with quantitative analysts and business stakeholders to translate requirements into production-grade systems. Support global production operations, monitoring, incident resolution, and data governance while using technologies such as Kafka, Spark, S3, Trino, and Starburst. Contribute to AI-powered monitoring agents and scalable analytics platforms.

Build and operate scalable batch, streaming, and event-driven data pipelines, cloud data platforms, trusted data products, and analytical models. Improve data quality, observability, reliability, performance, and cost efficiency while supporting risk, finance, product, analytics, and AI initiatives. Partner cross-functionally, influence architecture, lead complex technical initiatives, mentor engineers, and establish strong engineering practices including testing, CI/CD, governance, and infrastructure automation.

Designs and maintains enterprise healthcare EDI data pipelines for HIPAA X12 transactions, Medicaid encounters, and CMS compliance. Builds ETL/ELT workflows using Databricks, PySpark, Delta Lake, and SQL Server; develops validation, reconciliation, monitoring, and alerting frameworks; troubleshoots clearinghouse integrations; supports regulatory updates and encounter submissions; and collaborates with claims, compliance, security, product, and external healthcare teams.

Lead the architecture and evolution of enterprise-scale data platforms, applying distributed-systems and advanced data-modeling patterns. Establish technical standards across dbt, Airflow, Snowflake, Kafka, ETL, cloud infrastructure, security, monitoring, CI/CD, testing, and source control. Solve complex data problems, partner with senior leaders, provide technical governance, mentor engineers, and drive platform and process improvements.

Designs, builds, and operates governed AWS data lake infrastructure, self-service platform tooling, ETL/ELT pipelines, and data governance capabilities. Develops Python services supporting GenAI, RAG, and knowledge-based applications; contributes to streaming pipelines and large-scale PySpark processing. The role also manages infrastructure as code, observability, data quality, cost optimization, and platform reliability while partnering with engineering, data science, and business teams.

Own datasets for attribution, campaign performance, and customer lifecycle analysis. Design real-time and batch data pipelines, build scalable analytics infrastructure, develop attribution frameworks, optimize Oracle databases and SQL data models, and improve data governance. Partner with analytics and product teams to translate business needs into data solutions.

Build and lead the telemetry and analytics data platform, including collection, transformation, aggregation, data modeling, dashboards, and self-service tooling. Process OpenTelemetry traces, metrics, and logs at scale; analyze system performance, stability, and feature usage; ensure data quality and access; and partner with frontend and backend teams on instrumentation standards. Manage two data engineers, provide technical direction, document tooling, support users, and communicate findings through visualization.

Build and support production-grade Microsoft Fabric data products and pipelines across ingestion, transformation, lakehouse, warehouse, and curated layers. Implement reliable, restartable, and observable data solutions using medallion architecture, automated testing, governance, lineage, monitoring, CI/CD, and DataOps practices. Collaborate with architects, analytics, governance, application, and partner teams while documenting mappings, runbooks, controls, and operational procedures.

Designs and advances Providence’s enterprise data platform through data architecture, governance, metadata management, Snowflake modernization, legacy migrations, and scalable integration patterns. Establishes engineering, DataOps, CI/CD, repository, and data quality standards while supporting analytics, reporting, AI, and data products. Collaborates across multidisciplinary teams to implement large-scale data initiatives in a healthcare environment.

Lead data engineering strategy and develop enterprise data lake solutions on AWS. Build data pipelines using AWS data integration, engineering, and analytics resources, develop API services with FastAPI or Flask, optimize engineering practices, create long-term data plans, and represent the team in executive meetings.

Lead client data conversion and implementation projects for Defined Benefits plans. Analyze source data, identify and resolve data quality issues, coordinate with clients and internal stakeholders, and apply SQL, relational database, ETL, and Fidelity data-loading processes to deliver accurate migrations and corporate actions across multiple concurrent projects.

Designs, builds, and operates large-scale Hadoop data pipelines and analytics platforms. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark-based processing solutions; managing streaming platforms, distributed systems, data stores, and workflow orchestration; and supporting reliable production operations. The role also involves troubleshooting, debugging, documentation, and potentially cloud, lakehouse, Kubernetes, governance, CI/CD, and infrastructure-as-code technologies.

Leads the design, implementation, and support of a near-real-time transactional cache platform. Builds CDC pipelines from UniVerse to PostgreSQL, transforms data into optimized MongoDB models, and supports API-serving workloads on GCP. Owns performance, reliability, observability, incident response, monitoring, and operational support. Provides technical leadership, establishes engineering standards, drives modernization, collaborates with application teams, and mentors engineers across data platform initiatives.

Build and own production-grade data pipelines, source connectors, warehouse transformations, curated models, reconciliation, lineage, and data quality systems. Design scalable dbt projects and orchestration workflows using Airflow or Dagster, optimize warehouse performance, integrate messy enterprise systems, and ensure pipelines are reliable, idempotent, auditable, and cost-effective. Collaborate with backend and AI teams, establish data contracts, review work, and mentor engineers.

Build and own scalable data architecture, canonical data models, configurable source integrations, and reliable pipelines for AI-driven financial intelligence. Use SQL, Python, dbt, cloud data warehouses, and orchestration tools to deliver idempotent, auditable data workflows. Establish lineage, data contracts, quality controls, and schema evolution practices. Lead architecture reviews, mentor distributed data engineers, and support customer-hosted multi-cloud deployments with strict tenant isolation and traceability.

Build and maintain dbt and semantic data models in Snowflake, develop ingestion and transformation pipelines, define trusted product metrics, and improve data collection. Partner with Product and Engineering to support experimentation and feature analysis. Establish data quality testing, documentation, HIPAA-aware access controls, and compliance practices. Use and validate AI-generated code and Snowflake Cortex outputs while raising technical standards across the team.

Investigate and resolve complex data quality, integration, and reconciliation issues across ERP, e-commerce, databases, ETL pipelines, warehouses, and reporting systems. Trace discrepancies to their root causes, remediate and backfill data, improve master-data mappings, automate reconciliation and monitoring, document lineage and dependencies, and establish stronger controls. The role requires independent production troubleshooting, deep SQL Server and T-SQL expertise, cross-functional collaboration, and the ability to improve fragile legacy data processes.

Own the analytics data layer by translating stakeholder needs into certified dimensional models, governed metrics, semantic layers, dashboards, and reports. Build with dbt, SQL, Databricks, Power BI, and Airflow while ensuring data quality, observability, reconciliation, documentation, and dependable orchestration. Partner closely with finance, operations, and client-facing teams, automate recurring workflows, and use AI-assisted engineering practices in a fully remote environment.

Designs, builds, and operates large-scale Hadoop data pipelines and analytics platforms. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark-based processing workflows; working with Hadoop ecosystem technologies, streaming platforms, SQL, and distributed systems; orchestrating workflows; troubleshooting production environments; and supporting reliable, scalable data platforms for analytics, machine learning, and reporting.

Designs, builds, and operates large-scale Hadoop-based data pipelines and analytics platforms. Responsibilities include ingesting, transforming, and analyzing structured and unstructured data; developing Spark and streaming workloads; managing distributed systems, databases, and workflow orchestration; troubleshooting production platforms; and delivering reliable, performant, cost-effective data infrastructure for analytics, machine learning, and reporting.

Design, build, and operate production data pipelines using Python, Spark SQL, and Airflow. Canonicalize healthcare and customer data into trusted internal models, optimize distributed processing, enforce schemas, and implement quality checks and observability. Support customer onboarding, streaming pipelines, troubleshooting, on-call operations, and platform best practices while collaborating with product, analytics, engineering, data science, and customer teams.

Own end-to-end client data migrations: reverse-engineer legacy ETL, design and build Airflow DAGs and Spark jobs, validate and reconcile data at scale, document architecture and decisions, and coordinate cross-functional teams through go-live and hypercare.