Every engineering team has their own definition of success — and their own way of measuring it.
For some, there is one metric that shows quality above all else. Just ask Nathan Milford, vice president of infrastructure at Ro, a direct-to-patient healthcare company.
“Quality means low-friction systems where engineers can move fast without working around the system — the safe path is paved, observable and predictable by default,” Milford said.
For other teams, it’s a carefully balanced system of analytics; like at GC AI, an AI and legal tech company.
“There is not just one quality metric,” Chris Dzoba, engineer at GC AI said, explaining how the team uses a series of tests and AI performance scans. “The need for quality permeates all aspects of our development process... because quality is what establishes trust with the customer. Building that trust is critical.”
Built In spoke with engineering professionals across industries who shared an inside look at how their teams ship without chaos — and the metrics that prove their tools and processes are working.
Grainger is a distributor of maintenance, repair and operating products, serving more than 4.5 million customers worldwide.
What’s your rule for releasing fast — and what KPI proves it works?
Releasing fast starts with being intentional about how we reduce risk. We break work into smaller pilots, release incrementally and use what we learn to shape the next step. Operating this way allows us to stay close to customer feedback and maintain agility without letting releases become too large, too risky or too disconnected from the people actually using the technology.
Within our KeepStock function, this is important since our technology supports team members who are on site with customers every day, helping manage inventory and placing orders via their mobile app, sometimes in low-fi environments. The KeepStock technology roadmap is constantly evolving based on customer needs and team member feedback, so speed is not just about fast execution, it’s about staying close enough to the work so that we can respond when a customer or team member needs to shift.
The KPI I pay attention to is our responsiveness to change: how quickly we can introduce something new when there’s an opportunity and how quickly we can fix or roll back when something isn’t working. Mean time to recovery is a key signal there, but so is our ability to detect issues early and to take action before they become bigger problems. To me, that’s what proves we are moving fast without creating chaos.
Which standard or metric defines “quality” in your stack?
Quality in our stack is not one single metric; it’s the combination of reliability, resilience, performance and maintainability. We support a broad technology environment, which means we both integrate newer technology and modernize critical systems while supporting thousands of real-world customer interactions without disruption. Our systems need to operate in today’s state while also being able to evolve tomorrow.
From an operational standpoint, we look at things like system performance, mean time-to-recovery and whether we have the right signals in place to know when something is going wrong before it escalates to a more serious issue. If a system is slowing down, behaving unexpectedly or creating friction for team members in the field, we need the tooling and visibility to identify that quickly and respond accordingly. From an engineering standpoint, quality also shows up in the health of the code itself. We care about end-to-end testing, code quality and how easily the code can be changed. If code is difficult to understand or modify, that slows down future delivery, so quality is both customer-facing and developer-facing. The experience needs to be dependable for the people using the technology and the stack needs to remain durable for the teams building on it.
That’s where our team culture matters, too. We rely on consistent communication, pair programming and shared understanding, so teams are not solving complex problems in isolation. That collaboration helps us catch issues faster, make better decisions and keep moving the roadmap forward.
Name one recent AI/automation shipped and its impact on the team or business.
KeepStock has an ambitious roadmap and our teams support technology that directly serves customers. AI gives us another tool to improve the way we build, not by replacing engineering judgment, but by helping engineers move through complexity, synthesize information and focus more energy on higher-value problem solving.
One practical way we’re using AI is to help refactor and modularize code so teams can work in parallel more effectively. This may not sound flashy, but it can have a meaningful impact. When code is easier to understand and more modular, teams can reduce bottlenecks and work in parallel more effectively.
AI is also helping us synthesize information faster and understand how complex systems fit together. In a space like KeepStock, where we support a broad set of tools and capabilities, that kind of context-building is incredibly valuable. It can help engineers onboard faster, strengthen code quality checks and spend more time solving the highest-value problems.
What I appreciate about our approach is that we’re being thoughtful about leveraging AI. We’re creating space for teams to experiment, capture learnings and understand where AI genuinely helps. The goal is not to use AI everywhere all at once, but rather to use it where it has the highest impact: to improve how we build, how we learn and how quickly we can iterate to deliver meaningful outcomes for our customers.
Moov is a payments infrastructure platform making it easy to accept, store, send and spend money all from a single API.
What’s your rule for releasing fast — and what KPI proves it works?
At Moov, we're building the systems that software companies use to move money. That means every release has the potential to affect real financial transactions, so our definition of "shipping fast" is a little different.
Our rule is simple, we don't consider a feature "launched" just because it's deployed to production. We release new capabilities to a small group of pilot customers first.
When we launched support for card payments, for example, we started with two pilot customers before making it broadly available. That approach exposes real-world edge cases early while keeping the impact of any issues intentionally small. Shipping fast shouldn't mean taking unnecessary risks.
The KPI we care about isn't deployment frequency by itself. Its deployment frequency measured alongside change failure rate, how often a release introduces an issue that requires a fix or rollback. Anyone can ship often. The real test is whether the failure rate stays flat or even improves as release cadence increases. If it starts climbing, you've bought speed by introducing hidden risk. When you're building infrastructure that moves money, that's not a trade we're willing to make.
Which standard or metric defines “quality” in your stack?
Moov isn't a layer bolted onto someone else's payment processor. We connect directly to the card networks (Visa, Mastercard and American Express), the Federal Reserve for automated clearing house and FedNow and The Clearing House for real time payments. Owning those connections means owning the failure modes instead of inheriting someone else's black box.
But that's not how we define quality. The real test is whether a developer can integrate with Moov without becoming a payments expert. Concepts like “For Benefit Of” accounts, ISO 8583 card messaging, National Automated Clearing House Association file formats, reconciliation and network rules should stay behind our API, not become our customers' problem.
We measure that with our quality processing rate: the percentage of transfers processed correctly end to end. If a customer has to learn how National Automated Clearing House Association files or card authorizations work just to troubleshoot an integration, we've failed. Our customers don't care whether an issue started with our cloud provider or a payment network. To them, it's all Moov and that's the standard we hold ourselves to.
Name one recent AI/automation shipped and its impact on the team or business.
We had a project that had been fully scoped for almost a year. It touched four engineering teams and, by our estimates, would have required a dedicated five-person team working for roughly three months. It wasn't blocked because it lacked value; it simply kept losing priority to smaller projects that could be delivered more quickly.
In April 2026, we tackled it using our internal agentic coding pipeline instead. One engineer directed the work across four codebases from start to finish and the project was completed in 29 days. That's roughly one-third of the calendar time we'd originally estimated and more than a 90 percent reduction in engineering effort.
That's the outcome we're looking for with AI. We're less interested in whether it can generate a function a little faster and more interested in whether it allows one engineer to successfully deliver work that previously required pulling an entire team away from their roadmaps. The goal isn't to replace engineers. It's to remove the coordination overhead that keeps great engineers from tackling bigger problems.
Ro is a direct-to-patient healthcare company with a mission of helping patients manage their care journey.
What’s your rule for releasing fast — and what KPI proves it works?
Make the safe path the default path. Complexity creates drag, risk and cognitive load (for humans and agents). At Ro, we invest in paved-road tooling like Rodeo, our service framework, which bakes in primitives for observability, authentication, compliance, redundancy and scaling. The KPI is not just deployment throughput, rather, it is throughput cross-referenced with incidents and defects. Shipping faster only matters if we are not creating more cleanup work or breaking things for our patients in the process.
Which standard or metric defines “quality” in your stack?
Quality means low-friction systems where engineers can move fast without working around the system — the safe path is paved, observable and predictable by default. That translates into a better patient experience because teams catch issues earlier, recover faster and reduce the chance of defects reaching patients. Things will break, so we try to “score the glass” so they break in manageable ways. The standard is to make problems visible, containable and learnable.
Name one recent AI/automation shipped and its impact on the team or business.
In a broad, distributed system not everyone has high context on all parts of it. Ro has developed a Slack-based incident assistant that helps on-call engineers triage issues using Ro-specific context. It correlates deployments, anomalies, service ownership, architecture and product context from our observability and incident systems. The goal is simple: reduce the time engineers spend hunting for signals and help them get to the root cause faster for our engineering stakeholders and to ensure a seamless patient experience.
GC AI is an AI and legal tech company for in-house counsel. The platform enables legal professionals to be strategic business partners through faster and more accurate drafting, reviewing, researching and redlining.
What’s your rule for releasing fast — and what KPI proves it works?
The rule for releasing fast is: release fast. Release based on customer input. Release knowing you will iterate on most things, though occasionally you're going to have to be sure you've got the perfect solution the first time.
The KPI that proves it works is sales. If you're not making something that people want, they're not going to buy it. Ultimately business is driven by people's willingness to buy what you have made.
Which standard or metric defines “quality” in your stack?
There is not just one quality metric. We have all the standards: linting, type checks, unit tests, integration tests and end-to-end tests. We have standards that we follow for consistency. We have AI perform scans on what we build. We have peers review what we've built. We have internal testing, alpha testing, beta testing. After something is deployed we have monitoring to be sure things are running well. We have an external third party test our functionality and another third party test for security.
The need for quality permeates all aspects of our development process... because quality is what establishes trust with the customer. Building that trust is critical.
Name one recent AI/automation shipped and its impact on the team or business.
For internal use, a big level up was a skill we call "transcript to triage." The skill is used by an AI agent and integrates with the Granola MCP and the Linear MCP. The agent grabs the transcript from Granola, breaks each request into tasks, then searches Linear for those tasks. If there's a duplicate then the agent logs the ticket that this idea was mentioned again. If not, then the agent makes a new ticket.
Converting calls to tickets a few years ago was such a pain. It would take a long time and things would always fall through the cracks. This feels like magic.
Hanover Park is an AI-native fund administration company built to modernize financial services.
What’s your rule for releasing fast — and what KPI proves it works?
Merging is releasing. We deleted the release, no trains, no freeze windows. The main KPI is production and the unit of shipping is a stacked difference small enough to actually read. You ship a branch or three, then stack the next ones on top before the first is reviewed. Nobody sits blocked behind their own work.
Speed stays safe because the ceremony scales with blast radius. Low-risk visual changes ship on an adversarial AI audit alone. Features get AI review plus one human approval. Anything touching money gets a domain owner's deep read, every time, no exceptions. We spend rigor where mistakes are expensive and go flat-out everywhere else. If you need a release train to feel safe, you'll hate it here.
The KPI is land rate, straight from git. Other metrics include sixteen pull requests per engineer per week; median difference of 168 lines; median open-to-merge of two hours; weekly land rate is up two and a half times in two months, while the review bar got stricter. Small differences and matched rigor buy speed and safety at the same time.
Which standard or metric defines “quality” in your stack?
We do fund administration. Our output is other people's books: capital accounts, allocations, the statements Limited Partners make real decisions on. Test coverage is a weak standard for that. Coverage measures effort; we measure truth.
The standard is a verification ladder every engineer here knows cold. Typecheck proves shape, tests prove intent, the running system proves behavior and for money none of that is enough. Money needs an oracle: an independent path that computes the same number, or an invariant that cannot break. If no oracle exists for the number you're touching, building one is part of your task. Then decorrelated review: fresh eyes, human or adversarial AI, hunt for what the author can't see.
None of it lives in a Wiki. Canaries assert structural invariants across the ledger and they check for exactly zero dollars of drift. No tolerances.
Name one recent AI/automation shipped and its impact on the team or business.
Email agents. In fund administration the inbox is the API of the business: bills, wire notices, schedule-of-investment updates, Limited Partner contact changes. All of it lands as unstructured email that accountants used to triage by hand. Last quarter we shipped sandboxed LLM agents that read a firm's email, safely query that firm's live data as needed and draft the accounting task ready for one-click review.
We've built out the tools to allow agents to safely draft actions for our accountants to action on. They reach production data through a read-only Postgres role that cannot bypass row-level security: arbitrary SQL in, zero writes, zero cross-tenant reach. The database enforces that, not a system prompt.
Autonomy is earned. Every draft lands in front of an accountant and we manage one number, the no-edit rate — drafts approved without a single change. Accountants now review structured drafts instead of doing data entry and agents earn autonomy the same way engineers here do — with evidence.
