Top Data Engineer Jobs in Richmond, VA

Design, implement, and manage advanced data ecosystems and scalable ETL/ELT pipelines. Analyze complex problems, deliver data warehousing and governance solutions, mentor junior staff, and build client relationships while maintaining high standards and business-context awareness.

Lead design and implementation of enterprise data architecture and cloud-based data solutions. Manage large data engineering projects, develop data models and pipelines, ensure governance and security compliance, advise clients strategically, and build high-performing inclusive teams.

Designs, builds, administers, and supports cloud data architectures, databases, schemas, metadata repositories, ETL processes, and analytic models. Leads data quality, governance, troubleshooting, testing, automation, reporting, dashboards, and infrastructure design. Collaborates with Agile teams to translate business requirements into technical capabilities and supports decision-making for Audit and business management. The role is full-time onsite at an eligible Federal Reserve location and requires extensive security screening.

Administer and enhance Edmunds’ Databricks data platform and AWS infrastructure. Build and maintain ETL pipelines, infrastructure-as-code tooling using Terraform or CDK, and operational dashboards, alerts, and reports. Collaborate with business, engineering, analytics, security, and infrastructure teams to support data platform users and AI solutions. Evaluate new technologies, troubleshoot platform issues, and improve operational, cost, and security visibility.

Define and govern enterprise-scale data architecture across batch, streaming, warehouse, lakehouse, transactional, and AI use cases. Establish standards for data quality, lineage, access, cataloging, governance, observability, and SLAs. Architect AI-enabled workflows, resolve complex architecture issues, influence roadmaps, and mentor engineers through hands-on technical leadership. The role requires 15+ years of software, data engineering, or architecture experience and expertise in large-scale data platforms and modeling.

Lead data engineering initiatives by designing, developing, testing, implementing, and supporting cloud-based data solutions. Build distributed, real-time, streaming, and full-stack systems using Python, Spark, cloud platforms, relational and NoSQL databases, and data warehouses. Collaborate with Agile teams and product managers, perform code reviews and unit testing, optimize performance, stay current with technology trends, and mentor other engineers.

Leads data engineering initiatives by designing, developing, testing, implementing, and supporting cloud-based data solutions. Collaborates with Agile teams, product managers, and machine learning engineers; works with distributed systems, databases, streaming applications, and data warehouses. Performs code reviews and unit testing, optimizes performance, explores emerging technologies, and mentors engineering team members.

Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.

Staff Data Engineer will set technical direction and build Eve’s scalable data platform. Responsibilities include Snowflake administration, role-based access, governance, ingestion, dbt modeling and orchestration, Terraform infrastructure, observability, schema change detection, reliability practices, and CI standards. The role also mentors engineers and analysts, supports AI-assisted development, and establishes secure, reliable workflows for product, case, customer, and go-to-market data.

Own Eve’s data platform end to end, including ingestion, orchestration, incremental dbt models, Snowflake administration, Terraform infrastructure, observability, access controls, and pipeline reliability. Build schema-change detection, freshness SLAs, CI environments, and incident processes while managing performance and compute costs. Partner with analytics engineers and stakeholders to maintain trusted data for reporting and AI systems, using AI-assisted development and documenting platform standards.

Build and maintain Databricks-based data platforms, including ingestion, transformation, storage, governance, data modeling, and serving pipelines. Establish medallion architecture standards, canonical data models, quality controls, lineage, schema evolution, and reliable batch or incremental processing. Improve pipeline observability, scalability, idempotency, and recoverability while moving curated data to systems such as ClickHouse. Collaborate across application and analytics teams to create durable, governed production datasets.

Lead data engineering efforts by designing, developing, testing, and supporting scalable data pipelines and movement solutions. Build cloud-based systems using Python, SQL, Spark, Kafka, Snowflake, and Databricks; optimize data models, quality, governance, and performance; collaborate with Agile teams and product managers; conduct code reviews and testing; and mentor engineers while applying emerging technologies and interactive AI tools.

Leads technical strategy for enterprise data pipelines and data-sharing platforms. Builds scalable, resilient, high-performance systems using AWS, lakehouse architecture, Kafka, Flink, Spark, Snowflake, and Databricks. Develops code, drives engineering excellence and technology adoption, influences enterprise stakeholders, advises on platform capabilities, mentors engineers, and recruits technical talent. The role requires deep data engineering and architecture expertise, hands-on leadership, and innovation across internal and external data environments.

Design, build, and maintain scalable Spark-based ETL pipelines and computed tables in a central data lake. Integrate structured and unstructured IoT, sensor, and external data for analytics, model training, and dashboards. Collaborate with Data Science, Analytics, and ML teams to ensure reliable, high-quality customer-facing datasets.

Build and support scalable data platforms and pipelines, leading migrations such as Snowflake to BigQuery and transitioning reporting to Looker. Responsibilities include data architecture, ETL/ELT, API and marketing integrations, data quality, production troubleshooting, warehousing, performance optimization, and platform modernization. The role partners with analytics and business teams, owns projects through production, documents solutions, and provides technical guidance.

Configure and maintain Adobe Experience Platform and Real-Time CDP data pipelines, XDM schemas, datasets, identity resolution, Profile enablement, segmentation readiness, and destination activation. Validate data quality, troubleshoot ingestion and identity issues, manage sandboxes, and implement privacy, consent, governance, and access controls. Partner with architects, data engineers, consultants, analysts, data scientists, and marketing teams to support reporting, personalization, and machine learning readiness.

Lead development of scalable data pipelines and cloud-based data solutions using Python, SQL, Spark, Kafka, Snowflake, and Databricks. Responsibilities include data modeling, data quality, governance, testing, performance optimization, technical design, collaboration with Agile teams and product managers, and mentoring engineers. The role supports omnichannel marketing and messaging platforms while encouraging adoption of emerging technologies and interactive AI tools.

Design, build, and maintain scalable data pipelines, data models, and analytics solutions using Microsoft Fabric and Azure. Develop Lakehouse and Warehouse architectures, Fabric Notebooks with PySpark and Spark SQL, ETL/ELT workflows, and Power BI semantic models. Optimize performance, ensure data quality, security, governance, and regulatory compliance, and collaborate with analysts, data scientists, and business stakeholders to support business intelligence and advanced analytics.

Build and maintain large-scale data processing pipelines using Apache Spark (PySpark/Scala) and Python. Design ETL workflows, implement distributed data processing, use version control (Git), and work with cloud platforms (GCP/AWS/Azure).

Own production data ingestion pipelines for healthcare and benefits data, including normalization, mapping, validation, reconciliation, observability, and secure handling of sensitive information. Unify census data processes, scale the platform, improve identity matching, and build automation and TypeScript tooling. The role requires strong Python and SQL skills, production orchestration experience, reliable testing and data-quality practices, and effective communication with technical and nontechnical stakeholders.

Build and maintain Databricks data pipelines, ETL/ELT processes, SQL transformations, and Power BI dashboards for HR and People Analytics. Ensure data quality, optimize models and reporting performance, troubleshoot issues, document solutions, and collaborate with HR stakeholders and technical teams to deliver scalable workforce analytics.

Build and operate data-intensive web products and APIs, extend high-throughput ETL pipelines and scrapers, ensure reliability/observability, design self-scaling services, participate in code review, and collaborate across product, data, and infrastructure teams to meet production SLAs.

Build scalable, secure data pipelines, models, platforms, and governance systems that enable AI applications to access trusted enterprise data. Partner with analytics, marketing, and product teams to translate requirements into technical solutions, independently deliver projects through production, and support sensitive-data security and access controls. Preferred experience includes LLM architectures, cloud data platforms, infrastructure as code, regulated environments, and technical leadership.

Design and scale lakehouse architecture, including Bronze, Silver, and Gold data layers. Build reliable streaming and batch pipelines with Kafka, Spark, and Airflow; manage Iceberg, Delta, and Hudi table formats; optimize Trino, Starburst, and Databricks queries; and improve platform reliability, data quality, and performance. Collaborate with data scientists, analysts, and product teams to deliver scalable data platform solutions.

Design and scale lakehouse architecture, streaming and batch pipelines, and reliable data platforms using Kafka, Spark, Airflow, Iceberg, Databricks, and related technologies. Build Medallion-layer data systems, manage open table formats, optimize distributed queries, monitor platform reliability, resolve data quality issues, and collaborate with analysts, data scientists, and product teams.