This role owns quality for LOIS for Word, Filevine's AI-powered legal drafting product. You'll help build and own the QA function: the test strategy, the process, and the automation. You'll work at the intersection of legal AI and product quality, evaluating everything from AI output accuracy to UI behavior across the product. A big part of this role is moving quality upstream, so it's built into how features are planned and shipped from the start.
The ideal candidate has 2+ years of QA experience, brings a sharp eye for both functional bugs and AI quality issues, is comfortable building from scratch in ambiguity, and gets genuinely uncomfortable shipping something they haven't tried to break.
What You'll Do:
Define and own the test strategy, process, and QA playbook for the product
Move QA upstream by partnering with product and engineering early, so test plans are ready before features ship
Evaluate AI output quality across dimensions like accuracy, citation reliability, and appropriately scoped reasoning
Design and maintain evaluation sets that account for non-deterministic output and catch regressions over time
Test how AI-generated edits apply to documents, including accuracy and formatting fidelity across document structures
Build automated guardrails integrated into CI/CD that cover critical flows
Write structured, reproducible bug reports
Partner with legal domain experts to understand how different customer types use the product, and translate that into test coverage
Approach QA with a product and design mindset. If something works but could be better, we want to hear it
Compare behavior across product surfaces and document discrepancies
What You'll Need:
2+ years of QA experience, including ownership of test strategy for a product or major feature area
Track record of building QA process and automation from scratch in a fast-moving environment
Comfort operating in high ambiguity, where features ship quickly and you figure out how to test them fast
Strong analytical and problem-solving skills, with attention to both functional correctness and AI output quality
Ability to reason about non-deterministic systems and define what "correct" means when there's no single right answer
Excellent written communication, with the ability to write clear, concise, actionable bug reports backed by evidence
Experience with bug tracking tools and test management workflows
Nice To Have:
Experience testing AI or LLM-based products, especially for output accuracy and eval design
CI/CD experience setting up pipelines and release gates for automated tests
A design or UX sensibility, with an eye for where product workflows could be smoother
Prior experience in legal technology or comfort reading legal contracts
Programming experience beyond test automation
The salary range represents the low and high end of the salary range for this position. The total compensation package for this position will be determined by each individual’s location, qualifications, education, work experience, skills and performance.
An individual’s final offer will be weighted by their primary work location and expertise.
Skills Required
- 2+ years of QA experience
- Ownership of test strategy for a product or major feature area
- Strong analytical and problem-solving skills
- Experience with bug tracking tools
Filevine Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Filevine and has not been reviewed or approved by Filevine.
-
Healthcare Strength — Health coverage is described as covering the major bases (medical, dental, vision) and is often framed as decent quality. In some cases, premiums and copays are portrayed as relatively favorable, suggesting tangible value from the plans.
-
Parental & Family Support — Paid parental leave is positioned as a standard, clearly offered benefit. The presence of parental leave alongside disability coverage signals baseline family-support provisions typical of growth-stage tech employers.
-
Fair & Transparent Compensation — Compensation is sometimes framed as fair or reasonable relative to role expectations, with technical roles in particular appearing closer to market-aligned ranges. This creates pockets where pay is perceived as competitive even if not consistently top-of-market across the company.
Filevine Insights
What We Do
Filevine is case management software built for and inspired by real attorneys. As a fully-featured suite of tools, it comes ready to manage every part of a moving case. Assign tasks, upload files or images, monitor staff productivity, and communicate with your client directly from within their case file. Our software is built on the truth that every law firm functions differently. That’s why Filevine is so customizable. Build new case-type templates, design automatic workflows, and receive customized reports on a schedule that fits your needs. Accessing your information is never a problem, because Filevine is hosted on The Cloud. To ensure security, your law firm’s data is protected through state-of-the-art encryption on redundant servers. All you need to get started is an internet connection and your favorite web browser. Learn more at filevine.com.
Gallery
