Director, AI Quality & Reliability Engineering

Posted Yesterday
Be an Early Applicant
3 Locations
In-Office
118K-206K Annually
Expert/Leader
Analytics • Consulting
The Role
Lead and scale enterprise AI quality engineering across LLMs, agentic systems, and AI-enabled workflows. Build validation methodologies, runtime assurance, observability, testing automation, and governance-aligned processes. Partner cross-functionally, manage vendors, and develop high-performing teams to ensure reliable, compliant AI deployments at enterprise scale.
Summary Generated by Built In

When you’re the best, we’re the best. We instill an environment where employees feel engaged, satisfied and able to contribute their unique skills and talents while living and working as their authentic selves. We provide extensive opportunities for personal and professional development, building both employee competence and organizational capability to fuel exceptional performance through an inclusive environment both now and in the future.

Summary

In this role, you will lead operational execution and scaling of enterprise AI Quality Engineering capabilities across Vizient. Help operationalize scalable AI Quality Engineering frameworks, AI validation processes, runtime quality practices, AI-enabled testing approaches, and enterprise quality standards supporting responsible industrialization of AI-powered business solutions at enterprise scale. Combine quality engineering leadership, AI-enabled testing modernization, healthcare workflow validation, operational execution, organizational capability development, and cross-functional delivery leadership.

Responsibilities

  • Lead and scale enterprise AI Quality Engineering capabilities supporting AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.
  • Establish and operationalize AI-native quality engineering frameworks, validation methodologies, testing standards, runtime assurance practices, and scalable delivery processes.
  • Drive modernization of traditional quality engineering and test automation practices to support AI-enabled workflows, probabilistic systems, intelligent orchestration, and evolving healthcare business processes.
  • Lead AI validation strategies, including functional testing, prompt testing, workflow validation, regression testing, release readiness, runtime quality assurance, and production reliability.
  • Partner with AI Engineering, Data, Security, Governance, Clinical, and Operational teams to ensure quality, reliability, scalability, and successful deployment of enterprise AI solutions.
  • Implement quality controls, governance-aligned testing practices, validation evidence processes, and risk mitigation strategies that support responsible AI adoption and regulatory compliance.
  • Advance enterprise observability, telemetry, monitoring, and runtime quality practices to improve traceability, reliability, and confidence across AI-enabled systems.
  • Lead AI Quality Engineering delivery activities, including organizational planning, vendor and contractor management, execution governance, tooling strategy, and capability development.
  • Build, mentor, and develop high-performing quality engineering teams while fostering a culture of innovation, accountability, continuous improvement, and engineering excellence.
  • Research and evaluate emerging AI quality engineering, testing, automation, observability, and runtime assurance technologies to drive continuous improvement and organizational maturity.
  • Experience building and scaling organizational capabilities, operating models, and quality engineering centers of excellence.

Qualifications

  • 10 + years experience implementing quality engineering practices for Generative AI, Large Language Models (LLMs), agentic AI systems, or intelligent automation solutions.
  • Relevant degree, Bachelor’s degree preferred.
  • Experience working within healthcare, payer, provider, life sciences, or other regulated industries.
  • Familiarity with AI governance, responsible AI frameworks, model risk management, compliance requirements, and auditability practices.
  • Experience with AI-assisted testing tools, test automation platforms, observability solutions, telemetry platforms, and modern quality engineering technologies.
  • Knowledge of AIOps, LLMOps, MLOps, cloud-native platforms, and enterprise-scale AI deployment models.
  • Relevant certifications in Quality Engineering, Software Testing, Agile, Cloud Technologies, AI, or related disciplines.
  • Experience building and scaling organizational capabilities, operating models, and quality engineering centers of excellence.

#LI-JB1

Estimated Hiring Range:

At Vizient, we consider skills, experience, and organizational needs in our compensation approach. Geographic factors may adjust the range estimate and hires typically fall below the top range. Compensation decisions are tailored to individual circumstances. The current salary range for this role is $117,600.00 to $206,000.00.

This position is also incentive eligible.

Vizient has a comprehensive benefits plan! Please view our benefits here:

http://www.vizientinc.com/about-us/careers

Equal Opportunity Employer:   Females/Minorities/Veterans/Individuals with Disabilities

The Company is committed to equal employment opportunity to all employees and applicants without regard to race, religion, color, gender identity, ethnicity, age, national origin, sexual orientation, disability status, veteran status or any other category protected by applicable law.

Skills Required

  • 10+ years implementing quality engineering practices for Generative AI, Large Language Models, agentic AI systems, or intelligent automation solutions.
  • Experience working within healthcare, payer, provider, life sciences, or other regulated industries.
  • Familiarity with AI governance, responsible AI frameworks, model risk management, compliance requirements, and auditability practices.
  • Experience with AI-assisted testing tools, test automation platforms, observability solutions, and telemetry platforms.
  • Knowledge of AIOps, LLMOps, MLOps, cloud-native platforms, and enterprise-scale AI deployment models.
  • Experience building and scaling organizational capabilities, operating models, and quality engineering centers of excellence.
  • Leadership experience building, mentoring, and developing high-performing quality engineering teams.
  • Relevant degree (Bachelor's degree preferred).
  • Relevant certifications in Quality Engineering, Software Testing, Agile, Cloud Technologies, AI, or related disciplines.

Vizient Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Vizient and has not been reviewed or approved by Vizient.

  • Leave & Time Off Breadth Time off allowances are generous, with ample PTO and holidays available from day one. Separate paid volunteer days add additional protected time away.
  • Flexible Benefits Flexibility to work from anywhere for part of the year and hybrid options support work-life balance. Policies enabling remote periods complement the broader PTO structure.
  • Retirement Support Retirement benefits include a competitive 401(k) company match alongside HSA contributions on eligible plans. These elements strengthen long-term financial security within the total rewards package.

Vizient Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Irving, TX
5,661 Employees
Year Founded: 1977

What We Do

Vizient, Inc., the nation’s largest health care performance improvement company, serves more than 50% of the nation’s acute care providers, which includes 97% of the nation’s academic medical centers, and more than 20% of ambulatory care providers. Vizient provides expertise, analytics and advisory services, as well as a contract portfolio that represents more than $130 billion in annual purchasing volume. Vizient is based in Dallas and has offices in 20 metropolitan areas across the United States. We have 4,000 employees with a breadth of expertise, experience and compassion, who are eager to develop and implement solutions that advance health care for the greater good.

Similar Jobs

Boeing Logo Boeing

Procurement Multi-fam Mgr (Sup Mgt&Prcrmnt Multi Fam)

Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
In-Office
Dallas, TX, USA
170000 Employees
101K-173K Annually

Corporate Tools LLC Logo Corporate Tools LLC

Flutter App Developer

eCommerce • Legal Tech • Professional Services • Software • Data Privacy
Remote or Hybrid
4 Locations
1200 Employees
150K-150K Annually

Enverus Logo Enverus

Manager, Power Markets

Big Data • Information Technology • Software • Analytics • Energy
In-Office or Remote
5 Locations
1800 Employees
115K-130K Annually

MetLife Logo MetLife

Business Procedures Analyst

Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Remote or Hybrid
United States
43000 Employees
42K-72K Annually

Similar Companies Hiring

Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees
Scotch Thumbnail
Artificial Intelligence • eCommerce • Fintech • Payments • Retail • Software • Analytics
US
35 Employees
Milestone Systems Thumbnail
Artificial Intelligence • Security • Software • Analytics • Big Data Analytics
Lake Oswego, OR
1500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account