Senior AI Quality & Reliability Engineer

Posted 2 Days Ago
Be an Early Applicant
2 Locations
In-Office
102K-179K Annually
Senior level
Analytics • Consulting
The Role
Lead design and execution of AI Quality Engineering practices for LLMs and enterprise AI: build validation frameworks, automated testing, runtime observability, reliability engineering, and mentor teams while partnering across clinical, data, security, and product stakeholders to ensure responsible, reliable AI deployments in healthcare workflows.
Summary Generated by Built In

When you’re the best, we’re the best. We instill an environment where employees feel engaged, satisfied and able to contribute their unique skills and talents while living and working as their authentic selves. We provide extensive opportunities for personal and professional development, building both employee competence and organizational capability to fuel exceptional performance through an inclusive environment both now and in the future.

Summary

In this role, you will drive the hands-on design, development, and execution of AI Quality Engineering initiatives supporting Vizient's enterprise AI transformation efforts. You will design and implement AI Quality Engineering practices, AI validation processes, AI-assisted testing approaches, runtime quality controls, and scalable testing frameworks that support the responsible deployment of AI-powered business solutions.

This role combines hands-on quality engineering, AI-enabled testing modernization, healthcare workflow validation, technical mentorship, and cross-functional collaboration. You will help advance Vizient's Quality Engineering capabilities beyond traditional software testing toward AI-native validation, AI-assisted testing, runtime observability, reliability engineering, and modern AI Quality Engineering practices.

Responsibilities

  • Design, develop, and execute AI Quality Engineering strategies supporting AI-powered applications, large language model (LLM) solutions, intelligent automation, agentic systems, and enterprise AI platforms.
  • Build and implement scalable AI Quality Engineering practices, including AI-native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and automated testing frameworks.
  • Lead AI validation activities including functional testing, prompt validation, workflow testing, regression testing, release validation, runtime quality assurance, and production reliability support.
  • Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, Data, and Product teams to deliver scalable AI Quality Engineering processes across enterprise AI initiatives.
  • Support runtime reliability through observability, telemetry, distributed tracing, monitoring, drift detection, incident response, and operational quality assurance for AI-enabled systems.
  • Develop and maintain AI evaluation frameworks, validation datasets, quality scoring methodologies, and automated testing workflows that improve the reliability and scalability of AI solutions.
  • Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows, payer operations, and AI-enabled business processes while supporting responsible AI deployment through human-in-the-loop validation practices.
  • Coordinate testing activities across Agile delivery teams, including sprint planning, test execution, defect management, issue tracking, release readiness, risk identification, and production support.
  • Mentor Quality Engineers and provide technical guidance that promotes engineering excellence, AI-enabled testing modernization, continuous improvement, and adoption of modern Quality Engineering practices.
  • Research, evaluate, and recommend emerging AI Quality Engineering, testing automation, observability, and runtime assurance technologies to continuously improve enterprise AI quality capabilities.

Qualifications

  • Relevant degree preferred. Advanced degree in Computer Science, Engineering, or a related field highly preferred. Bachelor's degree in Computer Science,
  • 5 or more years of experience in Quality Engineering, Quality Assurance, software testing, enterprise application delivery, technology operations, or related technology functions required.
  • 2 or more years of experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects required.
  • Experience supporting Quality Engineering or Quality Assurance across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems required.
  • Strong knowledge of software development life cycle (SDLC), Agile methodologies, test automation frameworks, defect management, release validation, and production support processes required.
  • Experience validating AI-powered applications, intelligent automation, machine learning, large language model (LLM), or AI-enabled business workflows preferred.
  • Experience with AI Quality Engineering practices, AI-assisted testing, runtime observability, monitoring, telemetry, or reliability engineering preferred.
  • Strong analytical, problem-solving, organizational, communication, collaboration, and leadership skills required.
  • Demonstrated ability to manage multiple priorities and deliver results within fast-paced, highly collaborative enterprise environments required.
  • Experience in healthcare technology, payer operations, clinical workflows, or other regulated industries supporting AI governance and responsible AI deployment preferred.

#LI-JB1

Estimated Hiring Range:

At Vizient, we consider skills, experience, and organizational needs in our compensation approach. Geographic factors may adjust the range estimate and hires typically fall below the top range. Compensation decisions are tailored to individual circumstances. The current salary range for this role is $102,400.00 to $179,000.00.

This position is also incentive eligible.

Vizient has a comprehensive benefits plan! Please view our benefits here:

http://www.vizientinc.com/about-us/careers

Equal Opportunity Employer:   Females/Minorities/Veterans/Individuals with Disabilities

The Company is committed to equal employment opportunity to all employees and applicants without regard to race, religion, color, gender identity, ethnicity, age, national origin, sexual orientation, disability status, veteran status or any other category protected by applicable law.

Skills Required

  • Relevant degree (Bachelor's or advanced degree in Computer Science, Engineering, or related field)
  • 5 or more years experience in Quality Engineering, QA, software testing, enterprise application delivery, or technology operations
  • 2 or more years providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects
  • Experience supporting Quality Engineering or QA across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems
  • Strong knowledge of SDLC, Agile methodologies, test automation frameworks, defect management, release validation, and production support processes
  • Experience validating AI-powered applications, intelligent automation, machine learning, or LLM solutions
  • Experience with AI Quality Engineering practices, AI-assisted testing, runtime observability, monitoring, telemetry, or reliability engineering
  • Strong analytical, problem-solving, organizational, communication, collaboration, and leadership skills
  • Ability to manage multiple priorities and deliver results in fast-paced, collaborative enterprise environments
  • Experience in healthcare technology, payer operations, clinical workflows, or regulated industries supporting AI governance and responsible AI deployment

Vizient Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Vizient and has not been reviewed or approved by Vizient.

  • Leave & Time Off Breadth Time off allowances are generous, with ample PTO and holidays available from day one. Separate paid volunteer days add additional protected time away.
  • Flexible Benefits Flexibility to work from anywhere for part of the year and hybrid options support work-life balance. Policies enabling remote periods complement the broader PTO structure.
  • Retirement Support Retirement benefits include a competitive 401(k) company match alongside HSA contributions on eligible plans. These elements strengthen long-term financial security within the total rewards package.

Vizient Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Irving, TX
5,661 Employees
Year Founded: 1977

What We Do

Vizient, Inc., the nation’s largest health care performance improvement company, serves more than 50% of the nation’s acute care providers, which includes 97% of the nation’s academic medical centers, and more than 20% of ambulatory care providers. Vizient provides expertise, analytics and advisory services, as well as a contract portfolio that represents more than $130 billion in annual purchasing volume. Vizient is based in Dallas and has offices in 20 metropolitan areas across the United States. We have 4,000 employees with a breadth of expertise, experience and compassion, who are eager to develop and implement solutions that advance health care for the greater good.

Similar Jobs

Wipfli Logo Wipfli

Senior Manager, Accounting Advisory - Nonprofit Industry Clients

Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Remote or Hybrid
Naperville, IL, USA
3000 Employees
151K-203K Annually

Wipfli Logo Wipfli

Manager, Financial Reporting - FQHC Industry Clients

Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Remote or Hybrid
United States
3000 Employees
97K-145K Annually

Wells Fargo Logo Wells Fargo

Lead Equipment Finance Underwriter

Fintech • Financial Services
Hybrid
Hoffman Estates, IL, USA
205000 Employees
139K-260K Annually

Wells Fargo Logo Wells Fargo

Lead Equipment Finance Underwriter

Fintech • Financial Services
Hybrid
Chicago, IL, USA
205000 Employees
139K-260K Annually

Similar Companies Hiring

Amplify Platform Thumbnail
Fintech • Financial Services • Consulting • Cloud • Business Intelligence • Big Data Analytics
Scottsdale, AZ
62 Employees
Scotch Thumbnail
Artificial Intelligence • eCommerce • Fintech • Payments • Retail • Software • Analytics
US
35 Employees
Milestone Systems Thumbnail
Artificial Intelligence • Security • Software • Analytics • Big Data Analytics
Lake Oswego, OR
1500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account