Top Senior Data Engineer Jobs in Richmond, VA

Design, build, and operate scalable data transformation pipelines and dimensional models using dbt, SQL, and AWS. Implement infrastructure-as-code (AWS CDK), CI/CD, and query performance optimizations while ensuring data quality, governance, and collaboration with cross-functional and offshore teams.

Lead Data Engineer builds and maintains scalable data pipelines and lakehouse/warehouse platforms (Databricks) to support analytics, BI, reporting, and AI. Responsibilities include data modeling, ETL/ELT, medallion architecture, orchestration and monitoring (Airflow/Cron), performance tuning, CDC/incremental loads, data quality, and leading/mentoring a team while collaborating with stakeholders and maintaining documentation.

Design, build, and optimize scalable cloud data pipelines and infrastructure. Maintain and refactor SQL and ETL processes, automate data workflows, ensure data quality/security, and troubleshoot complex issues. Provide technical leadership and mentor junior engineers while collaborating with stakeholders to translate requirements.

Design, implement, and maintain PySpark-based data reconciliation solutions for financial systems. Build matching algorithms, integrate with rules engines, process large distributed datasets, identify and resolve discrepancies, and collaborate with analysts and architects to improve data quality and governance.

Design, build, and optimize PySpark applications and ETL pipelines to process large-scale datasets from SQL/NoSQL sources, data lakes, and streaming platforms. Ensure data quality, error handling, performance tuning, and collaborate with analysts, scientists, and architects to deliver scalable data solutions.

Design, implement, and maintain PySpark applications to automate large-scale financial data reconciliations. Build transformation and matching algorithms, integrate with rules engines, analyze data gaps, and collaborate with analysts and architects to ensure data quality and system resilience.

Design, build, and maintain scalable ETL/ELT pipelines into Snowflake, enforce data quality, optimize data models and query performance for Power BI reporting, produce documentation, and troubleshoot data issues while collaborating with stakeholders.

Design, develop, and maintain PySpark applications and ETL pipelines to process, transform, and integrate large-scale datasets from SQL, NoSQL, data lakes, and streaming sources. Optimize Spark job performance, implement robust error handling, and collaborate with data analysts, scientists, and architects using orchestration tools like Airflow or Luigi.

Design, develop, test, and deploy high-performance Spark/Scala data processing applications and ETL pipelines using the Cloudera Hadoop ecosystem. Optimize Spark and platform performance, ensure data integrity and security, collaborate with data scientists and analysts, troubleshoot issues, and implement version control and CI/CD for Spark applications.

Design and develop Big Data applications using Java, Spark, and MapReduce on Cloudera/Hadoop ecosystems. Work with Hive, Impala, YARN, Kafka, and large datasets; write tests (JUnit), perform data analysis, and script in Unix/Python. Lead and manage global technology teams and follow industry best practices.

Design and implement big data solutions (Spark, Hive, Java, CDP). Analyze and consolidate disparate data sources, produce functional specifications, review data models, gather stakeholder requirements, validate implementations, support production deployments, investigate data quality and data lineage, and collaborate with technology leads to ensure data completeness and accuracy.

Design and implement data federation and lakehouse architectures, build scalable ETL/ELT pipelines with Python and Spark, optimize performance across federated queries, manage Delta/Iceberg/Hudi tables, and enforce governance, security, and access controls for analytics and AI teams.

Design, build, test, and maintain high-performance Python data applications and backend services. Lead technical design, review code, mentor junior engineers, integrate with databases and cloud services, optimize performance, and ensure security, scalability, and reliability.

Design, develop, maintain, and optimize enterprise Tableau dashboards and BI solutions. Integrate and cleanse data from Oracle/Sybase/Big Data, tune SQL, automate Tableau extracts, manage Tableau Server on Linux, support testing, UAT, deployments, and troubleshoot data integrity and performance issues.

Own and build scalable ELT pipelines and event-driven ingestion in AWS, using Dagster for orchestration. Implement IaC (Terraform), data quality, lineage, monitoring, and troubleshoot cross-system issues while partnering with analytics and product teams to ensure downstream analytical correctness.

Architect, build, and maintain Snowflake-based ELT pipelines and production data models. Implement observability, data quality, and governance; optimize cost and freshness. Partner with Analytics, Product, and Engineering, use dbt/CI-CD and AI-assisted tooling, and participate in reviews and architecture planning to support analytics, BI, and ML.

Design, build, and maintain scalable, secure data pipelines and platforms to acquire, transform, and store large financial datasets. Enable AI/GenAI-ready data foundations for model training, inference, retrieval, and RAG workflows. Collaborate with cross-functional teams, ensure data quality, governance, observability, and compliance, and support production deployments and troubleshooting.

Support and maintain clients' Azure data platforms and Power BI environments. Design and deploy data pipelines, models, and analytics solutions using Microsoft stack (ADF, Synapse, Databricks, SSIS/SSAS/SSRS). Troubleshoot incidents, communicate with clients, manage licensing, document environments, and promote best practices while collaborating with stakeholders and continuous skill development.

Design, build, and operate secure, scalable cloud-native data platforms (lakehouse/data mesh) supporting batch, streaming, ML, GIS and operational workloads. Lead ingestion/transformation pipelines, distributed processing (Spark/Kafka), data governance, observability, and mentor teams while ensuring compliance in regulated environments.

The Senior Data Engineer will design scalable data processing frameworks using Java and Apache Spark, manage data pipelines, and mentor junior engineers.

Design and deploy scalable data infrastructure and models, maintain CDC pipelines, build reporting tools and dashboards, write high-performance SQL, partner with product and engineering to enable enterprise-wide analytics and data-driven decision making.

Responsible for designing, developing, and maintaining scalable data pipelines and infrastructure, implementing data governance, and collaborating with teams to meet data needs.

Design, build, and maintain data lakes, data warehouses, and end-to-end data pipelines for a multi-tenant SaaS platform. Lead code and design reviews, write infrastructure-as-code, optimize performance and scalability, participate in data governance, mentor junior engineers, evaluate new technologies, automate tests, collaborate cross-functionally, and join on-call rotation to ensure reliable data delivery.

The Senior Data Engineer designs and maintains data infrastructure, creating scalable data pipelines and ensuring data quality while collaborating with cross-functional teams.

Design, build, and maintain an internal data platform enabling secure, governed access for engineering and analytics. Partner with architects, implement data pipelines and automations, enforce security and compliance, support platform services, recommend architecture improvements, mentor teams, and drive adoption of scalable, self-service data capabilities.