Top Data Engineer Jobs

Develop data analytics and visual analytic applications using structured and semi-structured data. Design enterprise databases, build relational database systems, integrate data systems, and optimize data pipelines and performance. The role uses Java, AWS, Spark, Scala, SQL, PostgreSQL, NiFi, Informatica, Linux, and ETL technologies to support actionable intelligence applications. An active Top Secret/SCI-eligible clearance is required, with potential CI polygraph eligibility.

Build and operate production pipelines for legal records from PACER, court systems, opinions, PDFs, scans, XML, and HTML. Develop LLM-assisted extraction workflows, validate data quality, optimize SQL, and support RAG and semantic search products. Manage AWS S3 and RDS systems within security guardrails while collaborating with engineering teams. The role requires strong Python and SQL skills, legal data experience, document extraction expertise, and autonomous problem-solving with large, messy datasets.

Leads the design, implementation, and operation of scalable production data pipelines and architectures across cloud platforms. Defines technical approaches and success metrics with stakeholders, monitors reliability, mentors junior engineers, collaborates with data scientists and analysts, and develops reusable data engineering practices. The role requires production pipeline experience across multiple cloud providers, comfort in startup environments, and knowledge of regulated-industry compliance, security, and governance requirements.

Builds production-grade data pipelines, reusable feature stores, and scalable data products for AI and machine learning initiatives. Transforms enterprise data using Python, PySpark, SQL, Databricks, and Azure; develops features, validates data quality, manages governance and lineage, and optimizes workflows. Partners with AI/ML engineers and business stakeholders, supports experimentation and production troubleshooting, documents data assets, and contributes to engineering standards, security, and continuous improvement.

Leads the design and implementation of data engineering solutions, including robust pipelines, storage systems, data architectures, APIs, and data quality frameworks. Deploys solutions across Kubernetes, Teradata, AWS, and Databricks; selects appropriate storage platforms; maps data lineage; establishes development and QA standards; and collaborates with cross-functional teams to improve data accessibility, quality, and operational efficiency.

Lead the design and development of scalable data pipelines, microservices, cloud-native ingestion systems, and streaming solutions. Collaborate with architects, engineers, product teams, and business stakeholders to build reliable data platforms using Python, Java, AWS services, SQL, and relational and NoSQL databases. Establish data quality, testing, monitoring, automation, and validation frameworks while optimizing performance and resolving production issues across distributed systems.

Designs and optimizes enterprise data pipelines within Microsoft Fabric, including lakehouse and warehouse architectures, real-time ingestion, cloud migration, governance, CI/CD, and reporting layers. The role uses PySpark, Spark SQL, T-SQL, AI tooling, vector search, and RAG patterns while ensuring performance, reliability, and SLA compliance. Responsibilities also include mentoring junior engineers, leading technical standards, and collaborating with analysts, data scientists, and stakeholders.

Architects and builds enterprise data infrastructure connecting subscribers, content, advertising, and digital products. Leads distributed systems, APIs, event streams, real-time pipelines, Snowflake warehouses, dbt models, backend services, and machine-learning infrastructure for personalization. Establishes standards for observability, security, reliability, testing, and cost efficiency while providing technical leadership across AWS, containerized services, architecture, mentorship, and incident response.

Leads data engineering initiatives embedded with infrastructure teams. Designs governed data products, integrations, data contracts, quality controls, metadata, lineage, and standardized data models. Reconciles infrastructure asset inventories, resolves data discrepancies, and partners with engineering, product, and governance stakeholders to improve data completeness, accuracy, interoperability, security controls, and enterprise analytics.

Build and operate scalable data pipelines and platforms that transform LiDAR, imagery, and GPS sensor data into reliable geospatial deliverables. Responsibilities include automating ingestion, AI processing, quality checks, monitoring, workflows, and data access; partnering with Data Operations and GIS teams; and improving AWS processing performance and cost. The role requires end-to-end ownership, including system design, operation, and on-call support.

Build and operate scalable data pipelines and platforms that transform LiDAR, imagery, and GPS sensor data into reliable geospatial deliverables. Responsibilities include data ingestion, AI processing, quality monitoring, workflow automation, and AWS cost and performance optimization. The role partners with Data Operations, GIS, product, AI, and analytics teams and requires end-to-end ownership, including on-call support.

Architects, builds, tests, and maintains scalable data infrastructure and ELT/ETL pipelines using AWS Glue, Glue Catalog, and Step Functions. Performs data modeling, SQL querying, data migration, schema optimization, validation, and data quality improvements. Partners with analytics and business teams to transform raw data into production-ready datasets. The role may also support AI/ML pipelines, governance, access controls, DevOps workflows, and real-time data architectures.

Build and maintain data pipelines for network and operational telemetry, integrate DataMiner and other monitoring systems, create Power BI dashboards, configure monitoring metrics and alerts, automate reporting, analyze network performance and recovery events, and translate technical findings into actionable insights for technical and executive stakeholders.

Administer and optimize Oracle application databases while developing data models, warehouses, ETL pipelines, and analytics solutions across Oracle and Snowflake. Build scalable Python and SQL workflows, support Oracle Analytics Server, manage MongoDB use cases, automate monitoring, and implement CI/CD. Ensure data quality, governance, security, high availability, disaster recovery, and performance across enterprise data platforms.

Designs and maintains reliable data pipelines and processing systems for high-volume telecommunications and digital evidence data. Builds data infrastructure supporting AI models and analytics, develops schemas and data models, optimizes performance and scalability, and monitors data quality. Collaborates with AI, data science, and software engineering teams, documents architecture, and follows secure practices for sensitive law enforcement data.

Build scalable, governed analytical data products, semantic layers, reusable data models, certified datasets, and AI-ready analytics foundations. Perform validation, quality checks, optimization, documentation, and lineage management. Collaborate with Analytics, Product, Engineering, Data Engineering, and business stakeholders on project planning and metric governance. Improve self-service reporting, dashboard performance, automation, and data consistency while mentoring junior team members.

Build and maintain scalable data platforms, lakehouses, warehouses, ETL pipelines, Spark workflows, and AI/ML-integrated data solutions. Design dimensional models, implement governance and monitoring, optimize queries, integrate disparate data sources, and support KPI dashboards. Collaborate with engineering, product, DevOps, DBA, and security teams while mentoring engineers and analysts. Automate infrastructure and deployments using AWS, Terraform, and CI/CD practices.

Designs and maintains scalable data pipelines and architectures supporting AI and machine learning workloads for Army intelligence operations. The role develops data models, ensures data quality, availability, security, and governance, optimizes workflows for real-time analytics, integrates AI/ML solutions, and collaborates with internal and external stakeholders. Expertise with Palantir Foundry, programming, ETL, databases, and large language models is required, along with an active TS/SCI clearance and eligibility for TS/SCI with polygraph.

Leads mission data integration and engineering initiatives supporting SOUTHCOM’s Common Intelligence and Operational Pictures. Responsibilities include building ETL pipelines, ingesting and transforming data, managing Databricks, Maven Smart System, and War Data Platform solutions, developing machine learning and statistical models, validating data quality, supporting secure interoperability, collaborating with intelligence and operational teams, delivering training, and documenting analytical processes.

Build, maintain, and optimize ETL/ELT pipelines, data integrations, REST API connections, and SnapLogic workflows. Manage PostgreSQL queries, database objects, data quality, warehouse updates, automation, monitoring, troubleshooting, and performance optimization. Collaborate with stakeholders and vendors on reporting, regulatory FCC requirements, documentation, database upgrades, and analytics capabilities.

Design and build scalable cloud-native data platforms, batch and streaming pipelines, data lakes, and integrated data services. Support analytics and AI/ML workloads while implementing DevSecOps, infrastructure as code, and reliability improvements. Collaborate with data scientists, engineers, architects, consultants, and clients to solve complex data challenges and translate requirements into technical solutions. The role supports Defence, Government, and National Security projects and requires high-level security clearance and on-site work three to four days weekly.

Design, build, and manage scalable batch and real-time data pipelines, databases, and processing systems. Develop and administer Databricks and Snowflake environments, optimize data architecture and distributed processing, manage PostgreSQL and MySQL databases, and implement governance, monitoring, alerting, and troubleshooting practices. Collaborate with analytics, engineering, and product teams to deliver reliable data solutions, document workflows, and promote modern data engineering practices.

Build and operate scalable multimodal data pipelines from requirements through delivery. Own feasibility, planning, quality standards, annotation, processing, storage, cost tracking, and acceptance criteria. Coordinate engineers, annotation teams, vendors, and client-facing stakeholders while improving infrastructure, automation, tooling, dataset visibility, and data quality for AI and foundation model teams.

Leads the design and development of scalable, secure data architectures and production data pipelines. Builds batch and streaming ETL/ELT solutions, manages data quality and reliability, applies cloud-native technologies, and ensures compliance with privacy, residency, security, and financial regulations. Partners with technical teams and business stakeholders, defines technical strategy, evaluates technologies, and delivers software solutions. Preferred responsibilities include mentoring engineers, leading technical initiatives, and applying AI/ML technologies.

The role involves designing and building scalable data platforms, migrating data to the cloud, and implementing CI/CD practices while collaborating with cross-functional teams.