Lead Architect, AI Quality and Reliability Engineering

Posted 11 Days Ago
Be an Early Applicant
55435, Minneapolis, MN, USA
In-Office
135K-237K Annually
Senior level
Analytics • Consulting
The Role
Lead end-to-end quality engineering and validation for an AI-enabled healthcare operations analytics product. Design scalable automated validation frameworks, data quality and reconciliation, ontology and metric validation, LLM and AI evaluation, test harnesses, regression suites, and observability. Partner across product, engineering, and domain teams, mentor engineers, and drive root-cause resolution to ensure accurate, reliable, and traceable data, metrics, APIs, analytics, and user workflows.
Summary Generated by Built In

When you’re the best, we’re the best. We instill an environment where employees feel engaged, satisfied and able to contribute their unique skills and talents while living and working as their authentic selves. We provide extensive opportunities for personal and professional development, building both employee competence and organizational capability to fuel exceptional performance through an inclusive environment both now and in the future.

Summary:

In this role, you will lead Quality Engineering and product validation for an AI-enabled healthcare operations analytics product that integrates EHR, ERP, and other health system data to deliver standardized ontologies, operational metrics, analytics, and AI-powered workflows. You will serve as a hands-on technical leader, ensuring accuracy, reliability, traceability, and usability across the product lifecycle, from data acquisition and transformation through ontology and metric computation to analytics, AI-generated insights, and end-user workflows. You will establish scalable automation, validation frameworks, and quality engineering practices for solutions built on Palantir Foundry and AIP.

Responsibilities:

  • Lead the end-to-end Quality Engineering and product validation strategy across data pipelines, ontologies, metrics, APIs, analytics, AI capabilities, and user-facing workflows.
  • Design and implement scalable automated validation frameworks for data ingestion, transformations, ontology mappings, business rules, metric calculations, APIs, application workflows, and AI-generated outputs.
  • Establish data quality and reconciliation capabilities that validate completeness, accuracy, consistency, source-to-target mappings, lineage, and transformation logic across EHR, ERP, and other healthcare data sources.
  • Develop validation strategies for ontologies and operational metrics to ensure concepts, mappings, calculations, aggregations, and analytical outputs remain accurate and consistent as the product evolves.
  • Lead quality engineering for AI-enabled capabilities, including LLM evaluation, prompt and workflow testing, grounding and factuality validation, regression testing, agentic workflows, and human-in-the-loop evaluation.
  • Build reusable test harnesses, representative datasets, automated regression suites, quality gates, and observability capabilities that enable teams to deliver solutions with confidence.
  • Partner with Product, Software Engineering, Data Engineering, AI Engineering, and healthcare domain experts to translate product requirements into measurable acceptance criteria and drive complex quality issues through root cause analysis and resolution.
  • Advance a shift-left quality culture by mentoring engineers and establishing practices for testability, automation, data quality, reliability, and AI quality while remaining hands-on with critical product validation capabilities.

Qualifications:

  • Relevant degree preferred.
  • 7 or more years of relevant experience required.
  • Strong hands-on programming and automation skills using Python, Java, TypeScript/JavaScript, or similar languages, including experience building test frameworks, validation services, or engineering tooling.
  • Experience validating complex data pipelines, ETL/ELT processes, transformations, analytical datasets, APIs, business rules, and calculated metrics, with strong SQL and data analysis skills.
  • Experience designing automated data quality, reconciliation, source-to-target validation, regression testing, and end-to-end product validation approaches.
  • Experience testing modern data-intensive and full-stack applications and integrating automated testing and quality gates into CI/CD pipelines.
  • Experience evaluating AI-enabled applications using approaches such as LLM evaluation, prompt testing, RAG or grounding validation, and agentic workflow testing.
  • Experience with observability, telemetry, monitoring, cloud platforms, and modern DevSecOps practices.
  • Experience with Palantir Foundry/AIP or comparable enterprise data and AI platforms preferred.
  • Strong ownership, technical leadership, problem-solving, and collaboration skills, with the ability to operate effectively in ambiguity and influence cross-functional teams.

Estimated Hiring Range:

At Vizient, we consider skills, experience, and organizational needs in our compensation approach. Geographic factors may adjust the range estimate and hires typically fall below the top range. Compensation decisions are tailored to individual circumstances. The current salary range for this role is $135,200.00 to $236,600.00.

This position is also incentive eligible.

Vizient has a comprehensive benefits plan! Please view our benefits here:

http://www.vizientinc.com/about-us/careers

Equal Opportunity Employer:   Females/Minorities/Veterans/Individuals with Disabilities

The Company is committed to equal employment opportunity to all employees and applicants without regard to race, religion, color, gender identity, ethnicity, age, national origin, sexual orientation, disability status, veteran status or any other category protected by applicable law.

Skills Required

  • 7 or more years of relevant experience
  • Hands-on programming and automation using Python, Java, TypeScript/JavaScript, or similar languages
  • Strong SQL and data analysis skills
  • Experience validating complex data pipelines, ETL/ELT processes, transformations, analytical datasets, APIs, business rules, and calculated metrics
  • Experience designing automated data quality, reconciliation, source-to-target validation, regression testing, and end-to-end product validation approaches
  • Experience testing modern data-intensive and full-stack applications and integrating automated testing and quality gates into CI/CD pipelines
  • Experience evaluating AI-enabled applications (LLM evaluation, prompt testing, RAG/grounding validation, agentic workflow testing)
  • Experience with observability, telemetry, monitoring, cloud platforms, and modern DevSecOps practices
  • Strong ownership, technical leadership, problem-solving, and collaboration skills
  • Relevant degree
  • Experience with Palantir Foundry/AIP or comparable enterprise data and AI platforms

Vizient Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Vizient and has not been reviewed or approved by Vizient.

  • Leave & Time Off Breadth Time off allowances are generous, with ample PTO and holidays available from day one. Separate paid volunteer days add additional protected time away.
  • Flexible Benefits Flexibility to work from anywhere for part of the year and hybrid options support work-life balance. Policies enabling remote periods complement the broader PTO structure.
  • Retirement Support Retirement benefits include a competitive 401(k) company match alongside HSA contributions on eligible plans. These elements strengthen long-term financial security within the total rewards package.

Vizient Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Irving, TX
5,661 Employees
Year Founded: 1977

What We Do

Vizient, Inc., the nation’s largest health care performance improvement company, serves more than 50% of the nation’s acute care providers, which includes 97% of the nation’s academic medical centers, and more than 20% of ambulatory care providers. Vizient provides expertise, analytics and advisory services, as well as a contract portfolio that represents more than $130 billion in annual purchasing volume. Vizient is based in Dallas and has offices in 20 metropolitan areas across the United States. We have 4,000 employees with a breadth of expertise, experience and compassion, who are eager to develop and implement solutions that advance health care for the greater good.

Similar Jobs

CrowdStrike Logo CrowdStrike

Consultant

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
USA
11000 Employees
115K-160K Annually

CrowdStrike Logo CrowdStrike

Consultant

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
USA
11000 Employees
115K-160K Annually

CrowdStrike Logo CrowdStrike

Consultant

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Remote or Hybrid
2 Locations
11000 Employees
95K-140K Annually

The Aerospace Corporation Logo The Aerospace Corporation

Program Specialist II - Rotational program

Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Hybrid
3 Locations
4600 Employees
82K-103K Annually

Similar Companies Hiring

Northslope Thumbnail
Artificial Intelligence • Information Technology • Software • Analytics • Consulting • Generative AI
London, GB
100 Employees
Scotch Thumbnail
Artificial Intelligence • eCommerce • Fintech • Payments • Retail • Software • Analytics
US
35 Employees
Milestone Systems Thumbnail
Artificial Intelligence • Security • Software • Analytics • Big Data Analytics
Lake Oswego, OR
1500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account