AI Solutions Quality Assurance Analyst

Posted 11 Hours Ago
Be an Early Applicant
19 Locations
In-Office
93K-142K Annually
Mid level
Legal Tech
The Role
Own the full QA lifecycle for AI-powered legal solutions: design and run unit, integration, system, performance, and scalability tests; integrate automated test suites into Azure DevOps CI/CD; generate and evaluate AI-assisted test cases; track defects and quality metrics; and collaborate on security and reliability testing.
Summary Generated by Built In

Build your big career with the firm that does Big Law, Better. McDermott Will & Schulte is a leading global law firm that brings together more than 1,700+ lawyers and 1,400 business professionals. We celebrate excellence, collaboration, and community and have been recognized as a top workplace by USA Today, Fortune, The American Lawyer, Vault and others. We are also certified by Great Place to Work. 

At McDermott, we don’t just invest in your future, we accelerate your career – wherever it may lead. That includes supporting you both in and outside of the office. 

With us, you’ll find: 

  • A firm where everyone belongs: Our award-winning culture prioritizes warmth and authenticity — we encourage you to be yourself!  

  • Enthusiasm for all perspectives: We’re smarter and stronger when everyone has a voice and a seat at the table. We welcome unique viewpoints and ideas, and we make opportunities for you and your career to thrive.  

  • Support to feel your best and do your best: Wellness is integral to building a successful career and a rich life. That’s why our benefits program supports your physical, emotional, mental, and financial health, with an emphasis on work-life balance.  

  

  • Real rewards for real work: We offer generous compensation packages that recognize hard work and excellence.   

Job Description:

McDermott, Will & Schulte LLP is building one of the most ambitious AI programs in the legal industry — and quality is not an afterthought. As we deploy AI-powered solutions across legal practices and operational teams, we need someone who can own the full quality lifecycle: from unit tests and edge cases through system testing, performance, and CI/CD pipeline integration. The AI Solutions QA Analyst partners directly with our Analysts and engineering team to ensure every solution we ship is tested, reliable, and production-ready — leveraging AI itself to do it faster and more thoroughly than traditional QA ever could.

WHY MWS:

  • Shape quality standards for an entirely new category of legal AI solutions — from the ground up 

  • Work with frontier AI tools to build and run tests faster than traditional QA methods allow 

  • High-visibility role reporting to Director of QA with direct impact on every solution the team ships 

  • Remote-first, technically rigorous team that takes production quality seriously 

WHAT YOU'LL DO:

  • Own the full QA lifecycle for AI-powered solutions — from unit and integration testing through system, performance, and scalability testing 

  • Partner with Analysts to generate and evaluate test cases using AI-assisted tools, with particular focus on boundary conditions, edge cases, and AI output validation 

  • Work with the engineering team to define and integrate automated test suites into the Azure DevOps CI/CD pipeline for each solution 

  • Define system-level test strategies in collaboration with the Director of QA, covering functional, regression, and non-functional testing requirements 

  • Coordinate with the dedicated security team to translate security testing guidance into solution-specific security test cases 

  • Evaluate AI solution reliability across varied inputs — ensuring consistent, predictable behavior under real-world conditions 

  • Maintain test documentation, defect tracking, and quality metrics that give leadership clear visibility into solution readiness 

  • Champion a quality culture across the delivery team — making testing a natural part of the development workflow, not a gate at the end 

WHAT YOU BRING: 

  • 4+ years of formal QA experience, including writing and executing test scripts across the full testing lifecycle 

  • Strong experience with unit, integration, system, and regression testing — both manual and automated 

  • Hands-on experience integrating test suites into CI/CD pipelines; Azure DevOps experience strongly preferred 

  • Demonstrated ability to design boundary and edge case tests for complex, data-driven applications 

  • AI fluency — you use AI tools actively and can direct leading AI platforms to generate, evaluate, and refine test cases 

  • Understanding of performance and scalability testing concepts; experience with load testing tools a plus 

  • Solid grasp of defect tracking, test documentation, and quality metrics reporting 

  • Strong analytical thinking — you approach untested AI behavior the way an engineer approaches an unknown system 

  • BS in Computer Science, Information Systems, Engineering, or equivalent experience 

 

Preferred: 

  • Experience testing AI-powered applications — prompt-driven systems, agentic workflows, or LLM-integrated tools 

  • Familiarity with AI-assisted test generation workflows and code-generating platforms 

  • Exposure to security testing principles in collaboration with a dedicated security team 

  • Experience in legal, financial services, or other regulated industry environments 

  • Knowledge of Azure DevOps, including pipeline configuration and automated test integration 

Successful candidates will be provided with outstanding career opportunities and will receive a competitive total rewards package with the opportunity to earn performance-based bonuses.
    

Target Hiring Range $93,000 - $142,000

Please note that quoted salary ranges are not guarantees of what final salary offers may be. Base pay is based on market location and may vary depending on job-related knowledge, skills, experience, and geographic location. Base pay is only one part of the Total Rewards that MWE provides to compensate and recognize our staff professionals for their work. Full time positions are eligible for a discretionary bonus and a comprehensive benefits package. 

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

#LI-JL1 #LI-Hybrid

Physical Demands and Work Environment:

The physical demands and work environment characteristics described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

Physical Demands:

  • While performing the duties of this job, the employee is required to sit, use hands, reach with hands and arms, stoop, talk and hear

  • Employee must occasionally lift up to twenty (20) pounds

Work Environment:

Typical indoor office environment

Disclaimer:

The above statements are intended to describe the general nature and level of the work being performed by people within this classification. They are not intended to be an exhaustive list of all responsibilities, duties and skills required of employees assigned to this job.

 

 

Skills Required

  • 4+ years of formal QA experience
  • Experience writing and executing test scripts across unit, integration, system, and regression testing
  • Hands-on experience integrating test suites into CI/CD pipelines
  • Azure DevOps experience
  • Ability to design boundary and edge case tests for complex, data-driven applications
  • AI fluency; use AI tools to generate, evaluate, and refine test cases
  • Understanding of performance and scalability testing concepts
  • Experience with load testing tools
  • Solid grasp of defect tracking, test documentation, and quality metrics reporting
  • Strong analytical thinking
  • BS in Computer Science, Information Systems, Engineering, or equivalent experience
  • Experience testing AI-powered applications (prompt-driven systems, agentic workflows, LLM-integrated tools)
  • Familiarity with AI-assisted test generation workflows and code-generating platforms
  • Exposure to security testing principles in collaboration with security teams
  • Experience in legal, financial services, or other regulated industry environments
  • Knowledge of Azure DevOps pipeline configuration and automated test integration

McDermott Will & Schulte Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about McDermott Will & Schulte and has not been reviewed or approved by McDermott Will & Schulte.

  • Strong & Reliable Incentives Bonus programs often match or exceed market, with discretionary and special awards that materially raise total compensation in strong years. Merit-driven structures are seen as enhancing overall satisfaction among attorneys.
  • Fair & Transparent Compensation Compensation communications and public announcements outline explicit base and bonus levels by year, reinforcing clarity around pay expectations. Market-scale adjustments and stated increases support perceptions of fairness.
  • Wellbeing & Lifestyle Benefits Well-being initiatives include 24/7 mental-health access, individual coaching, mindfulness resources, and billable credit for certain activities that integrate wellness into work. On-site counselor support in select offices and app-based tools further extend day-to-day support.

McDermott Will & Schulte Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Chicago, IL
3,597 Employees
Year Founded: 1934

What We Do

McDermott Will & Emery is now McDermott Will & Schulte. Leading organizations turn to global law firm McDermott Will & Schulte for a better way to address legal challenges, connect with those at the forefront, and drive stronger outcomes. Working across more than 20 offices globally, our 1,750+ lawyers act on data-driven insights, deep relationships, and unmatched industry experience to deliver on our commitment of Always Better.

Similar Jobs

PNC Bank Logo PNC Bank

Product Manager

Machine Learning • Payments • Security • Software • Financial Services
Remote or Hybrid
USA
55000 Employees
134K-250K Annually

PNC Bank Logo PNC Bank

Software Engineering Manager

Machine Learning • Payments • Security • Software • Financial Services
Remote or Hybrid
USA
55000 Employees
100K-223K Annually

Liberty Mutual Insurance Logo Liberty Mutual Insurance

Inside Sales Representative

Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Remote or Hybrid
10 Locations
40000 Employees
55K-75K Annually

Liberty Mutual Insurance Logo Liberty Mutual Insurance

Inside Sales Representative

Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Remote or Hybrid
9 Locations
40000 Employees
55K-75K Annually

Similar Companies Hiring

Eve Thumbnail
Legal Tech • Software • Generative AI
San Mateo, CA
180 Employees
GC AI Thumbnail
Artificial Intelligence • Legal Tech
San Mateo, California
100 Employees
Legora Thumbnail
Artificial Intelligence • Legal Tech • Software
Chicago, Illinois
700 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account