Engineers at Homebot have a game plan when it comes to measuring success.
For Bret Doucette, senior software engineer, it’s all about making sure that the metrics are measuring the right things to ensure a launch is successful.
“We don’t have a single metric that defines ‘quality,’ and I’d be skeptical of any team that says they do,” Doucette said. “For us, quality shows up as a combination of signals.”
Doucette and the engineering team at Homebot use dozens of signals to closely monitor the health of databases, latency and data throughput.
Built In spoke with Doucette in detail about how the real estate tech company measures success on the engineering team.
Homebot is a real estate tech company that empowers consumers with personalized and actionable financial insights through the full homeownership life cycle.
What’s your rule for fast, safe releases — and what KPI proves it works?
Our rule is pretty basic: Did something break immediately after we shipped? Six months ago, we had zero production monitors in Datadog. Today, we have over 70, covering critical endpoints, databases, latency, throughput, background jobs and AWS plus GCP infrastructure. A release is considered “safe” if no monitor fires post-deploy. If it does, we catch it before a customer does. The primary KPI is our Datadog monitoring coverage.
In other words, are we watching the parts of the system that matter? We validate that with a few outcome signals: Are we detecting issues from dashboards, monitors and notebooks instead of customer reports? How many revert pull requests do we need after deployments? Are error rates trending up? We also have predefined skills, combined with Datadog’s MCP tooling, to analyze issues and surface craziness faster. Our speed comes from having enough visibility to move confidently.
Which standard or metric defines “quality” in your stack?
We don’t have a single metric that defines “quality,” and I’d be skeptical of any team that says they do. For us, quality shows up as a combination of signals. First and foremost, did anything break after we shipped? Are error rates trending up? Did a Datadog monitor fire? Are we getting angry emails from customers? On the product side, we also look at Amplitude to see if people are actually engaging with the feature the way we expect. On top of that, we still do manual QA, especially for high-risk or client-facing changes. If those signals are quiet and the feature is being used as intended, we consider that high quality. Taken together, this is our definition of quality.
Name one recent AI or automation that shipped and its impact on the team or business.
How can I name just one? We’re building GitHub Actions that, as part of our CI/CD pipeline, automatically create Datadog notebooks and use Datadog’s MCP tooling to monitor deployments in real time, with the goal of opening pull requests to fix bugs as they are detected. Ideally, every feature is observable from the moment it ships, with monitors and runbooks generated automatically alongside the code monitored mostly by bots but with direction from humans. This lets us ship more often and rely more on automated PR reviews, with humans making high-level decisions. It also helps us stay confident we didn’t break anything in this fast-moving, Opus-induced agentic world.
What is a project that an engineer at Homebot might work on?
“We’re building GitHub Actions that, as part of our CI/CD pipeline, automatically create Datadog notebooks and use Datadog’s MCP tooling to monitor deployments in real time, with the goal of opening pull requests to fix bugs as they are detected.”
— Bret Doucette, Senior Software Engineer
On the review side, we built Rampaging Raccoons. It’s a multi-perspective PR review system for Claude Code. It dispatches multiple agents (security, performance, testing, etc.), merges their feedback and posts a single, human-style review that only flags issues the difference actually introduced. We also trigger targeted “skills” during CI, things like Sidekiq patterns and database migration best practices, so the review is grounded in how we actually build, not generic and often incorrect advice.
