We are seeking a highly skilled and experienced VP, Quality Engineering Lead to define, build, and drive our automated data and report testing strategy. In this role, you will lead the Quality Engineering (QE) initiatives for our next-generation, AI-powered data and reporting ecosystem.
As a hands-on leader, you will design robust automated test suites to validate complex data architectures—specifically focusing on data virtualization, massive data federation, data contract testing, and the verification of emerging natural language/conversational AI query interfaces. You will manage a talented team of quality engineers, establish testing standards, and collaborate closely with engineering, and product teams to ensure high-quality, secure, and performant data and report delivery.
Key Responsibilities1. Test Strategy & Quality Leadership- Data & Reporting Test Strategy: Architect and execute a comprehensive, end-to-end automated testing strategy covering data virtualization, federated queries, BI/reporting, and AI-enabled analytical interfaces.
- Team Leadership: Lead, mentor, and functionally manage a specialized team of Data & Report Quality Engineers, fostering a culture of modern Quality Engineering (QE) and continuous improvement.
- Governance & Compliance: Define operating standards, automated quality gates, and data verification protocols across the analytics and reporting delivery lifecycle.
- Stakeholder Management: Own the reporting of quality metrics, pipeline coverage, and test automation maturity to senior global technology and engineering leaders.
- Federated Query & Virtualization Validation: Develop automated testing frameworks to validate query execution, latency, and data integrity across massive federated query engines and data virtualization platforms (e.g., Starburst, Trino, Presto, Denodo, Dremio, AWS Athena, or Apache Drill) connecting dozens of heterogeneous catalogs without physical data movement.
- Data Contract & Schema Validation: Implement automated schema validation and data contract testing to ensure curated, virtualized data products strictly adhere to published business definitions and system requirements.
- Access Control & Security Testing: Design data-driven security tests to verify that centralized data access governance (e.g., Apache Ranger, role/attribute-based access controls) and data masking are flawlessly applied.
- Natural Language Query Testing: Establish frameworks to test conversational AI interfaces that allow users to query data using natural language. Validate natural language processing (NLP) models, intent recognition, NLP-to-SQL translation logic, and the accuracy of the underlying datasets returned.
- Autonomous Agent Verification: Design testing patterns for non-deterministic AI agents (e.g., automated alerting systems and contextual research assistants), validating logical outputs, threshold actions, and boundary limits.
- Report & Dashboard Verification: Devise automated strategies to test visual correctness, performance, and backend data reconciliation for BI platforms (e.g., Tableau, custom web-based dashboards) during large-scale migration phases of legacy systems (comprising hundreds of reports).
- Data Lakehouse & Pipeline Testing: Lead automation efforts validating complex data pipelines across hybrid databases (Oracle, SQL Server) and modern analytical lakehouses.
- Data Reconciliation: Design and automate source-to-target data reconciliation, schema drift detection, and data lineage validation to ensure reports match underlying source systems perfectly.
- Continuous Quality Pipelines: Seamlessly integrate data and report automation suites into enterprise CI/CD pipelines (Jenkins, Tekton, GitLab, etc.) to trigger continuous verification with each deployment code path.
- Triage & Defect Management: Champion structured defect triage, prioritizations, and root cause analysis across complex, multi-tiered data and reporting infrastructure environments.
- Data Virtualization & Federation: Hands-on experience with enterprise data virtualization or query federation platforms, such as Starburst, Trino, Presto, Denodo, Dremio, AWS Athena, or Apache Drill.
- BI & Reporting Platforms: Deep expertise in testing BI and reporting platforms (e.g., Tableau, custom web-based dashboards, Aspose, or similar reporting engines).
- Database Querying & Testing: Advanced SQL expertise with hands-on experience testing relational databases (Oracle, SQL Server) and NoSQL databases.
- Programming Languages: Proficiency in Python or Java to build, maintain, and scale custom test automation frameworks.
- API Testing: Strong experience with API testing (REST/SOAP) and data contract validation using tools like Postman, RestAssured, or custom scripts.
- CI/CD Integration: Experience integrating automated data test suites into enterprise CI/CD pipelines (e.g., Jenkins, Tekton, GitLab CI) to enable continuous testing.
- Test Methodologies: Deep understanding of Agile/Scrum methodologies, functional, integration, regression, and parallel-run testing for large-scale migrations.
- Modern Lakehouse & Warehouse Platforms: Familiarity with cloud-native data platforms, such as Databricks or Snowflake (experience with Google BigQuery is also valued).
- Distributed Data Processing Engines: Familiarity with distributed compute engines, specifically Apache Spark (PySpark, Spark SQL) or Apache Flink for large-scale data processing.
- Data Quality Automation: Experience implementing automated data quality frameworks using industry-standard tools such as Great Expectations, dbt test, Soda / SodaCL, or Deequ / PyDeequ.
- AI/ML & NLP Testing: Experience testing LLM-backed applications, validating Natural Language-to-SQL engines (e.g., conversational query interfaces), prompt validation, and autonomous agent testing.
- Agile QE Leadership: Strong experience running QA cycles within Scrum/Kanban frameworks, managing sprint closures, and collaborating with cross-functional Dev/Product leads.
- Test Strategy Design: Proven track record of designing multi-layered testing strategies (unit, integration, regression, system, and regression parallel runs for migrations).
- Total Testing Experience: Minimum 10-12 years of relevant experience in software testing, quality engineering, or data engineering.
- Data Automation Experience: Minimum 5-8 years of hands-on experience in automated data testing, ETL testing, or data pipeline quality engineering.
- BI & Analytics Verification: Minimum 5-8 years of experience in report/BI testing, data reconciliation, and source-to-target data validation.
- Education: Bachelor’s degree in Computer Science, Information Systems, or equivalent engineering field.
------------------------------------------------------
Job Family Group: Technology------------------------------------------------------
Job Family:Applications Development------------------------------------------------------
Time Type:Full time------------------------------------------------------
Most Relevant Skills Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Skills Required
- Hands-on experience with enterprise data virtualization and federation platforms (Starburst, Trino, Presto, Denodo, Dremio, AWS Athena, Apache Drill).
- Experience testing BI and reporting platforms (Tableau, Aspose, custom dashboards) including visual correctness and backend reconciliation.
- Advanced SQL expertise and hands-on testing experience with relational databases (Oracle, SQL Server) and NoSQL databases.
- Proficiency in Python or Java to build and maintain test automation frameworks.
- API testing and data contract validation experience (Postman, RestAssured, or custom scripts).
- Experience integrating automated data/report test suites into CI/CD pipelines (Jenkins, Tekton, GitLab CI).
- Design and implement automated schema validation, data contract testing, and source-to-target reconciliation.
- Experience designing data access/security tests and working with data governance tools (e.g., Apache Ranger), RBAC/ABAC, and data masking.
- Minimum 10-12 years total experience in software testing, quality engineering, or data engineering.
- Minimum 5-8 years hands-on automated data testing, ETL testing, or data pipeline quality engineering.
- Minimum 5-8 years experience in BI/report testing, data reconciliation, and source-to-target validation.
- Bachelor's degree in Computer Science, Information Systems, or equivalent engineering field.
- Familiarity with cloud data platforms (Databricks, Snowflake, Google BigQuery).
- Familiarity with distributed processing engines (Apache Spark / PySpark, Spark SQL, Apache Flink).
- Experience with data quality automation tools (Great Expectations, dbt test, Soda / SodaCL, Deequ / PyDeequ).
- Experience testing LLM-backed applications, NLP-to-SQL engines, and autonomous AI agent verification.
Citi Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Citi and has not been reviewed or approved by Citi.
-
Healthcare Strength — Benefits coverage is positioned as comprehensive, including health, dental, and vision insurance plus on-site clinics, prescription drug support, and disability coverage. Family-building support such as fertility assistance is described as a notable differentiator within the overall package.
-
Retirement Support — Retirement benefits are framed as strong, highlighted by a 401(k) with matching and additional plan options like a Roth 401(k). Financial support is reinforced through discounts and broader financial guidance resources tied to the benefits ecosystem.
-
Wellbeing & Lifestyle Benefits — Wellbeing support extends beyond insurance through programs like an Employee Assistance Program, counseling/legal resources, and gym or wellness reimbursement. These offerings increase the perceived total rewards value even when cash compensation sentiment varies by role.
Citi Insights
What We Do
Citi's mission is to serve as a trusted partner to our clients by responsibly providing financial services that enable growth and economic progress. Our core activities are safeguarding assets, lending money, making payments and accessing the capital markets on behalf of our clients. We have 200 years of experience helping our clients meet the world's toughest challenges and embrace its greatest opportunities. We are Citi, the global bank – an institution connecting millions of people across hundreds of countries and cities.
