Top Data Engineer Jobs in Philadelphia, PA

Designs, builds, administers, and supports cloud data architectures, databases, schemas, metadata repositories, ETL processes, and analytic models. Leads data quality, governance, troubleshooting, testing, automation, reporting, dashboards, and infrastructure design. Collaborates with Agile teams to translate business requirements into technical capabilities and supports decision-making for Audit and business management. The role is full-time onsite at an eligible Federal Reserve location and requires extensive security screening.

Build, optimize, and support scalable batch and streaming data pipelines for enterprise workforce, billing, and interaction datasets. Develop cloud-native solutions with AWS, Databricks, PySpark, Kafka, Kubernetes, and Airflow while processing tens of terabytes of data. Provide production support, performance tuning, troubleshooting, and data platform automation. Collaborate across Product, Governance, Analytics, and Engineering teams, support data warehousing and reliability practices, participate in architecture decisions, and mentor junior engineers.

Lead design and implementation of enterprise data architecture and cloud-based data solutions. Manage large data engineering projects, develop data models and pipelines, ensure governance and security compliance, advise clients strategically, and build high-performing inclusive teams.

Architect, deploy, and operate core data platform services (Airflow, Superset, Terraform). Build CI/CD pipelines, automate with Python, manage cloud infrastructure, implement security and compliance, develop dbt data models, catalog assets in OpenMetadata, and mentor junior engineers.

Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.

Design, build, and maintain scalable Spark-based ETL pipelines and computed tables in a central data lake. Integrate structured and unstructured IoT, sensor, and external data for analytics, model training, and dashboards. Collaborate with Data Science, Analytics, and ML teams to ensure reliable, high-quality customer-facing datasets.

Configure and maintain Adobe Experience Platform and Real-Time CDP data pipelines, XDM schemas, datasets, identity resolution, Profile enablement, segmentation readiness, and destination activation. Validate data quality, troubleshoot ingestion and identity issues, manage sandboxes, and implement privacy, consent, governance, and access controls. Partner with architects, data engineers, consultants, analysts, data scientists, and marketing teams to support reporting, personalization, and machine learning readiness.

Build, model, and maintain scalable BigQuery-based data solutions on GCP. Implement performant data models, storage partitioning/clustering, ETL improvements, data validation, and documentation. Collaborate with architects, data scientists, and engineers to deliver governed, high-quality data for reporting and AI initiatives.

Design, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, Snowflake, and Azure technologies. Build data ingestion frameworks, dimensional data models, and enterprise warehouse solutions. Optimize Snowflake workloads, implement data quality and governance controls, monitor production pipelines, troubleshoot incidents, document data processes, and collaborate with architects, analysts, data scientists, and business stakeholders. Support secure, compliant, reliable data platforms for analytics, reporting, and operational use cases.

Design, build, and operate scalable batch, streaming, and CDC data pipelines across Snowflake and AWS. Lead ETL/ELT orchestration, data modeling, quality controls, monitoring, observability, incident response, and performance optimization. Develop production-grade SQL and Python solutions, establish engineering standards, and partner with analytics, data science, product, and engineering teams to deliver reliable data products. Provide technical guidance and independently solve complex data platform problems.

Provides technical leadership for Finance, Actuarial, and Corporate Data platforms. Defines engineering strategy and standards, architects automated data pipelines, leads cloud migrations and ETL modernization, and drives AI-assisted development, DevOps, CI/CD, testing, and data quality practices. Manages senior data engineers and distributed teams, resolves complex engineering issues, governs technical delivery, and communicates strategies to executive stakeholders. Requires extensive enterprise data engineering, cloud platform, data warehousing, automation, and people leadership experience.

Develops scalable, secure ETL pipelines, database architectures, and real-time or batch data solutions. Integrates large datasets, ensures data quality and integrity, writes complex code and queries, and translates business needs into technical designs. Leads solution development, testing, releases, documentation, training, and vendor issue resolution while advising business partners on data analysis and tools.

Designs, builds, administers, and troubleshoots cloud data architectures, databases, schemas, metadata repositories, ETL processes, and analytic models. Ensures data integrity and quality, develops SQL queries and reports, leads testing and automation improvements, and supports dashboards, governance, data sharing, and metric collection. Collaborates with Agile teams to translate business requirements into technical infrastructure designs and decision-support capabilities using AWS, Java, Python or Scala, GitLab, and Terraform.

Build and maintain cloud cost monitoring, alerting, and optimization tooling. Develop automation, cost allocation/tagging, data pipelines, dashboards, and integrations with billing APIs. Support FinOps processes, implement cost guardrails, troubleshoot cost systems, participate in on-call rotation, and collaborate with senior engineers and product teams to drive cost-efficient infrastructure and reporting.

Support and improve enterprise data platforms by monitoring and troubleshooting ETL workflows, resolving production incidents, conducting root cause analysis, deploying enhancements, coordinating source-system changes with vendors and technical teams, tuning database and server performance, managing capacity, and maintaining operational documentation and stakeholder communications.

Designs and develops complex cloud-native data solutions, scalable batch and real-time pipelines, APIs, data models, and enterprise data platforms. Implements data governance, quality, lineage, security, observability, CI/CD, automated testing, and operational reporting capabilities. Collaborates with architects, product owners, analysts, governance, security, and platform teams on technology strategy and cloud modernization. Mentors engineers, leads technical design discussions, presents recommendations, and supports AI-ready data products and emerging development tools.

Design, build, and maintain Keboola-based data pipelines integrating multiple business systems to deliver accurate, scalable data for supply chain, logistics, and transportation analytics; collaborate with analysts and engineers; support documentation, governance, and pipeline automation.

Own the design, development, maintenance, and performance of Microsoft Fabric data pipelines, centralized semantic models, DAX measures, and documentation. Ensure data quality, consistency, and trusted marketing metrics across sources. Partner with data engineering and BI teams on deployments and production issues, investigate model discrepancies, perform analytical work, mentor apprentices, and contribute to applied AI initiatives.

Designs and scales cloud-native data platforms and ingestion pipelines using AWS, Databricks, and Spark. Builds APIs, file-upload mechanisms, ETL/ELT workflows, data transformations, and Lakehouse architecture supporting analytics, machine learning, and operational use cases. Provides operational support for legacy C# .NET ingestion processes, troubleshoots pipelines, and promotes data governance, security, reliability, and documentation in a multi-tenant SaaS environment.

Supports, maintains, troubleshoots, and optimizes production Databricks applications, Spark jobs, SQL queries, Delta Lake pipelines, clusters, workflows, and data quality processes. Responsibilities include platform configuration, observability, documentation, deployment support, technical testing, finance data engineering, stakeholder coordination, and incremental improvements. The role also contributes to reporting enablement, operational readiness, process improvement, and on-call support during EST business hours.

Designs and operates batch, streaming, and real-time data platforms and pipelines using Spark, Kafka, Iceberg, Airflow, and cloud infrastructure. Owns data lake evolution, data quality, observability, reliability, governance, event instrumentation, schema management, and privacy controls for PII, retention, deletion, and residency. Leads complex initiatives end-to-end, troubleshoots production issues, mentors engineers, and develops AI agent harnesses that enforce data engineering standards and quality gates.

Design and lead scalable data engineering solutions across Databricks and Microsoft Azure. Define enterprise data modeling standards, architect pipelines and integrations, establish engineering best practices, troubleshoot complex platform issues, and guide modernization and cloud migration initiatives. Provide technical leadership and mentorship while ensuring data platforms meet security, quality, governance, reliability, performance, and maintainability standards.

Develop and maintain automated data pipelines, API-driven workflows, and data integrations using Python, SQL, Databricks, and data lake architectures. Improve data quality through validation, schema design, and provider data mapping across diverse sources. Build analytics-ready datasets for predictive analysis, Power BI reporting, and GIS-based access analyses. Document data flows and technical configurations, support knowledge transfer, and collaborate with analysts, product owners, contractors, and government stakeholders.

Builds and maintains scalable data pipelines, ETL infrastructure, data warehouses, and analytics tools. The role assembles complex datasets, automates processes, optimizes data delivery architecture, and supports cross-functional data initiatives. Collaborates with executive, product, data, and design stakeholders to resolve technical issues and improve analytics systems. Requires proficiency in SAS, Python, Java, MATLAB, SQL, NoSQL, Hadoop, MapReduce, Hive, Presto, UNIX, and Linux, with JavaScript, R, and basic machine learning preferred.

Build and maintain enterprise data pipelines and infrastructure supporting natural-language querying, semantic search, knowledge graphs, and GraphRAG for the SEC. Responsibilities include onboarding data sources, processing XBRL and financial filing data, managing metadata and lineage, validating data quality, monitoring pipeline health, resolving integration issues, and using AWS data services. The role also requires compliance with federal security standards and collaboration with the Data Architect.