Top Data Engineer Jobs

Designs and deploys scalable data infrastructure, schemas, pipelines, data-quality monitoring, warehouse architecture, and Power BI solutions. Partners with product, engineering, marketing, sales, data science, and machine learning teams to define data requirements, support self-service analytics, productionize models, and deliver data-driven customer experiences. Uses SQL, Python, PySpark, cloud platforms, and AI technologies to transform raw data into reliable analytical and product datasets.

Build and operate governed data pipelines using Microsoft Fabric, Azure, Python, PySpark, Spark, and SQL. Implement Medallion Architecture, ingest data from APIs and enterprise systems, and develop lakehouse and Azure SQL data products supporting analytics and AI. Use Git and CI/CD for deployments, explore AI agentic workloads, and partner with architects and stakeholders to translate requirements into scalable solutions. The role requires local travel, occasional overtime and weekends, and in-person work flexibility.

Designs and maintains scalable data pipelines and architectures for operational, intelligence, and mission data. Develops dashboards, visualizations, data models, and AI-enabled analytics using Python, Maven Smart System, Palantir, and Power BI. Performs data integration, cleansing, scraping, exploitation, and predictive analysis while supporting C-UAS operations, assessments, acquisition decisions, and strategic planning. Produces technical documentation and executive briefings, ensures data governance and security, and collaborates with mission stakeholders.

Supports testing, troubleshooting, documentation, automation, data quality, and user acceptance testing for data pipelines and learning management system integrations. Builds or updates dashboards and reports using business intelligence tools, assists with ETL/ELT and data synchronization, and contributes to data cleanup and quality control. Collaborates with technical and business teams while communicating findings and supporting real-time enterprise data initiatives.

Leads end-to-end data enablement for Supply Chain and Manufacturing domains. Partners with business and technical stakeholders to define use cases, roadmaps, data models, semantic layers, integration requirements, quality controls, and governed data products. Builds and optimizes Databricks lakehouse pipelines using SQL, Python, PySpark, and Delta Lake across complex ERP, manufacturing, procurement, inventory, and finance systems. Establishes standards for lineage, security, documentation, reliability, and production readiness.

Designs and maintains scalable data pipelines, models, integrations, and ETL/ELT solutions using cloud data technologies. Analyzes complex data, implements quality controls and automated testing, troubleshoots data issues, documents data flows, and supports governance. Collaborates with product owners, stakeholders, architects, data scientists, and engineers in an agile environment while applying AI-assisted development responsibly.

Build and operate production data pipelines integrating ERP/ATS/project management/telemetry into a queryable layer. Own the data model for a live knowledge graph. Ship SLA-backed datasets/services for tools and ML. Convert unstructured vendor/field data (PDFs, spreadsheets) into trusted structured inputs. Ensure data quality via tests, monitoring, and lineage, and use modern data stacks and LLM tools responsibly.

Designs, builds, and maintains the ASPCA’s enterprise data warehouse ecosystem. The role develops dbt Core models across medallion architecture, integrates MDM workflows, builds and operates Microsoft Fabric and Azure Data Factory pipelines, and supports Snowflake environments. Responsibilities include dimensional modeling, orchestration, monitoring, testing, troubleshooting, performance optimization, documentation, CI/CD, data governance, and quality assurance. The engineer collaborates with BI, data management, strategy, and research teams to deliver trusted, analytics-ready datasets.

Build and support Databricks Lakehouse pipelines, curated datasets, and analytics-ready data models using Spark, Delta Lake, Python, and SQL. Integrate Databricks with SQL Server, SaaS, and enterprise systems; implement data quality, governance, security, and monitoring practices; optimize Spark workloads; and support CI/CD and platform administration. Collaborate with analysts, data scientists, stakeholders, and Data Enterprise teams to modernize legacy data processes and deliver reliable production data products.

Supports secure data operations within the SUNet environment, including ingestion, storage, transfers, validation, documentation, access management, troubleshooting, and data lifecycle activities. Maintains inventories and technical records, coordinates Change Requests and user onboarding, manages cloud and on-premises storage, and develops scripts for automation. The role supports Department of Defense AI and autonomy initiatives while collaborating with government, cybersecurity, engineering, vendor, and AI/ML stakeholders.

Develop and operate data pipelines supporting a federal analytics and business intelligence platform. Responsibilities include ingesting, processing, storing, and securely transferring data; deploying models; automating ingestion; maintaining applications; improving pipeline reliability; and implementing API-first, version-controlled, tested solutions in an Agile environment. The role collaborates with data scientists, architects, engineers, product managers, and federal stakeholders.

Designs and implements secure, scalable data infrastructure for intelligence weapons systems. Responsibilities include translating requirements into data models and pipelines, integrating multiple data sources, monitoring and optimizing workflows, documenting data processes, and supporting data architecture. The role requires advanced SQL, Python, data modeling, pipeline operations, and collaboration with government customers and cross-functional teams.

Designs, develops, and optimizes scalable batch and near-real-time data pipelines and platforms using Snowflake, AWS, Python, DataStage, and Control-M. Builds data models, ensures data quality and governance, tunes performance and costs, supports production operations, and enables analytics and AI/ML use cases. Collaborates with engineering, data science, BI, platform, and business teams in Agile environments.

Develop and maintain data pipelines, scalable data models, analytics datasets, and Power BI reporting solutions for renewable energy construction and preconstruction operations. Integrate cost, labor, project, scheduling, and operational data to support benchmarking, forecasting, estimate accuracy, and predictive analytics. Collaborate with estimators, engineers, procurement, project controls, operations, and leadership while using Databricks, Python, SQL, GitHub, and AI-enabled development practices.

Lead data engineering activities for mission-driven client projects by designing, developing, deploying, and maintaining scalable data platforms and ETL/ELT pipelines. Oversee data assessment, processing, and analytics solutions across structured and unstructured data. Mentor multidisciplinary teams, supervise deliverables, and support agile development, monitoring, and platform operations.

Develop data analytics and visual analytic applications using structured and semi-structured data. Design enterprise databases, build relational database systems, integrate data systems, and optimize data pipelines and performance. The role uses Java, AWS, Spark, Scala, SQL, PostgreSQL, NiFi, Informatica, Linux, and ETL technologies to support actionable intelligence applications. An active Top Secret/SCI-eligible clearance is required, with potential CI polygraph eligibility.

Build and operate production pipelines for legal records from PACER, court systems, opinions, PDFs, scans, XML, and HTML. Develop LLM-assisted extraction workflows, validate data quality, optimize SQL, and support RAG and semantic search products. Manage AWS S3 and RDS systems within security guardrails while collaborating with engineering teams. The role requires strong Python and SQL skills, legal data experience, document extraction expertise, and autonomous problem-solving with large, messy datasets.

Leads the design, implementation, and operation of scalable production data pipelines and architectures across cloud platforms. Defines technical approaches and success metrics with stakeholders, monitors reliability, mentors junior engineers, collaborates with data scientists and analysts, and develops reusable data engineering practices. The role requires production pipeline experience across multiple cloud providers, comfort in startup environments, and knowledge of regulated-industry compliance, security, and governance requirements.

Builds production-grade data pipelines, reusable feature stores, and scalable data products for AI and machine learning initiatives. Transforms enterprise data using Python, PySpark, SQL, Databricks, and Azure; develops features, validates data quality, manages governance and lineage, and optimizes workflows. Partners with AI/ML engineers and business stakeholders, supports experimentation and production troubleshooting, documents data assets, and contributes to engineering standards, security, and continuous improvement.

Leads the design and implementation of data engineering solutions, including robust pipelines, storage systems, data architectures, APIs, and data quality frameworks. Deploys solutions across Kubernetes, Teradata, AWS, and Databricks; selects appropriate storage platforms; maps data lineage; establishes development and QA standards; and collaborates with cross-functional teams to improve data accessibility, quality, and operational efficiency.

Lead the design and development of scalable data pipelines, microservices, cloud-native ingestion systems, and streaming solutions. Collaborate with architects, engineers, product teams, and business stakeholders to build reliable data platforms using Python, Java, AWS services, SQL, and relational and NoSQL databases. Establish data quality, testing, monitoring, automation, and validation frameworks while optimizing performance and resolving production issues across distributed systems.

Designs and optimizes enterprise data pipelines within Microsoft Fabric, including lakehouse and warehouse architectures, real-time ingestion, cloud migration, governance, CI/CD, and reporting layers. The role uses PySpark, Spark SQL, T-SQL, AI tooling, vector search, and RAG patterns while ensuring performance, reliability, and SLA compliance. Responsibilities also include mentoring junior engineers, leading technical standards, and collaborating with analysts, data scientists, and stakeholders.

Develop and maintain scalable data pipelines and secure infrastructure for multimodal synthetic data, AI evaluation, real-time model monitoring, and clinical research data. Responsibilities include data transformation, preprocessing, quality assurance, database management, privacy compliance, interactive dashboards, pipeline documentation, and scientific dissemination.

Architects and builds enterprise data infrastructure connecting subscribers, content, advertising, and digital products. Leads distributed systems, APIs, event streams, real-time pipelines, Snowflake warehouses, dbt models, backend services, and machine-learning infrastructure for personalization. Establishes standards for observability, security, reliability, testing, and cost efficiency while providing technical leadership across AWS, containerized services, architecture, mentorship, and incident response.

Leads data engineering initiatives embedded with infrastructure teams. Designs governed data products, integrations, data contracts, quality controls, metadata, lineage, and standardized data models. Reconciles infrastructure asset inventories, resolves data discrepancies, and partners with engineering, product, and governance stakeholders to improve data completeness, accuracy, interoperability, security controls, and enterprise analytics.