The product strategy you made at the end of 2019 didn’t work. It most likely was invalidated by the events of 2020 — and if it wasn’t, email me because I need to buy lottery tickets soon. As our industry and society have been forced to adapt to new circumstances, the importance of flexibility has become clear. Despite this, one of the things I’ve consistently heard from several teams and product managers I’ve coached/advised is that, although they understand the need to be flexible with projects, they don’t have a way to do so.
Often, broaching the subject of a pivot brings up a bunch of questions from everywhere in an organization. These questions can quickly shift to blame. Next thing you know, what began as a conversation about making a strategic change in a product’s development turns into a political sniping session where the stakes can be extremely high.
I’ve seen this happen at companies big and small. I once saw a project with a nine-figure budget implode all due to a small shift in direction, and it ended up taking almost two years to get to a shippable product. Similarly, at a 100-person company, a debate over one feature led to people threatening whole departments. Yikes.
As a result of these stakes, most teams would rather continue down the road they’re on and launch something, even though the product may no longer be relevant to the market. This rigidity means companies spend an untold amount of money on both development costs and opportunity costs. It’s easier to stick to the way you’ve always done things than it is to change. But that doesn’t change the fact that you can’t afford to waste resources.
Product management’s role is to improve their organization’s ability to make intelligent decisions. Like it or not, it falls on us to enable the company to change direction in a way that serves both the customers and the business itself.
A good change in direction should have three core qualities:
- Fast. The sooner we can change direction when things aren’t working, the better.
- Politically safe. The sooner we can remove the possibility of sniping and get aligned, the better.
- Data-informed. The sooner we can have some data to back our decisions, the better.
Fortunately, I’ve created a framework that helps PdMs take that pivot from just a thought to reality: survival metrics. Using these will help get your team unstuck and on the road to making better decisions.
What Are Survival Metrics?
Survival metrics help a product team determine if an initiative is worth investing in more, pivoting or stopping completely. They are a forcing function to prevent product teams from suffering due to the sunk cost fallacy. Survival metrics put resource allocation and company incentives, both implied (think politics) and direct (think data), in front of the team before a project begins and again at regular intervals, giving permission to act quickly.
Survival metrics create a clear picture of what can go wrong while a project is in motion. By spelling out potential limitations early, you’ve created a warning system for both the team and the company that will make any necessary pivots more effective since you won’t spend time convincing the organization to get on board with your change.
Survival metrics have three levels, each based on the information you’ll get from answering questions about the project. If something is worth stopping the project for, the metrics are STOP statements. Those that lead us to reconsider the direction we’re going in are PIVOT statements. Anything that signals you to lean into the initiative as you build, even if things are a little rocky, is an INVEST statement.
As Marty Cagan likes to remark, a product manager’s job is tied to the feasibility of an initiative. Is this worth doing based on our current state? Survival metrics help simplify those feasibility markers and make them public in a way that is readable to the rest of the organization.
Here are three questions that help you identify those survival metrics:
- What markers do we need to see to commit more resources to this initiative? Example answer: “If we get X metric, it’s worth adding more resources, because it’s important to our company’s identity/position/adoption, etc.”
- What are some non-negotiables for the team/company? Example answer: We have to get this feature out or else we’ll be sacked.
- How much are we willing to let this initiative cost us? Example answer: We can afford to put this out if we can do it with three engineers, one designer and one PdM in six months.
You can turn the answers to these questions into metrics, both quantitative and qualitative, through your intuition as product manager. You’ll find some of the answers through documentation, some of them through asking questions of the team members on the ground and some of them through talking to the stakeholders who own the profit and loss of the initiative. You’ll also find some of this data is already available internally in different departments. This information generally isn’t disclosed to the whole organization for many reasons, yet it still affects projects. Part of your work here is collecting it all.
Questions like the ones you saw above will give you a path into finding out how the organization ticks and will help you get to a place where any pivots are fast, politically safe and data-informed. So, spend some time talking to the teams around you. You can do this either informally during a coffee meeting/lunch or more formally with a survey.
Gathering this data can be difficult at first, but it’s vital for creating good survival metrics. When you get this information, it is critical that you translate and prioritize the answers that aren’t direct and then make everything public. By doing so, you’ve turned an implicit agreement about what factors will make the team change direction explicit for everyone to see.
Let’s see how this works through an example.
Introducing Bobco
Imagine you’re a new product manager for a company called Bobco, and the company is looking to make a new chat feature. You begin surveying the various departments that will play a role in building the feature. Almost immediately, you hit pay dirt. You find out from the engineering team that they have had a tough time with their Amazon Web Services spend and will blow past their budget if this tool leads to a 10 percent increase in cost.
The survival metric in this case is simple. If the AWS bill is projected to jump more than 10 percent due to a production release, you will need to discuss ways to lower the projected bill immediately or else the company will not be profitable. This limit may not have been discussed outside of operations meetings due to the ops teams thinking it’s irrelevant to other departments. In fact, you know this information is incredibly important to the team working on the feature. So, you’ll create a survival metric in the form of a PIVOT statement: “If the AWS bill is projected to jump more than 10 percent due to a production release, we will need to discuss ways to lower the projected bill immediately.”
Through your conversations, you also find out all of the following:
Bobco doesn’t use third-party info. This becomes a STOP statement: “If we have to grab information from third-party sources to complete our chat feature, scrap the project and invest the resources elsewhere.” Bobco considers this value non-negotiable, so put it here in the metrics, in plain sight of the entire company.
You talk to the leadership team, and they want to be sure that the problem they’re trying to solve is truly a big pain point for users. This will become a STOP statement: “If the problem definition research is not aligned in a reasonable way after we introduce a scenario, it could mean this problem isn’t clear to our users, and we’re wasting our time.”
You discover that the marketing department feels unclear about the product’s features, which becomes a PIVOT statement: “If we haven’t demonstrated the features to our marketing team in a way that they feel comfortable with the product, take some time to discuss process.”
As you talk to people across the organization, you hear repeatedly that the company doesn’t want to replicate products like those already on the market. This can become a PIVOT statement: “Bobco isn’t interested in making a chat app like Intercom due to market pressure. If our chat app is mimicking Intercom and users bring it up during our usability testing, we should reconsider the project.”
Your investigation shows that beta invites are likely to be greater in number than initially projected. This should be an INVEST statement: ”If our beta invite population is twice our current projection, this could signal real excitement from our customers. We should take time to talk about ways we could expand this project.”
These metrics will be both qualitative (the uncountable, from interviews and observations from your conversations with various departments like the research metric) and quantitative (the countable, like surveys and usage data from things like AWS metric).
If anything is important enough to change the status of the project, such as a team violating the values of a company, going over the budget allotted or even something that would call for more investment like a ton of usage, then it should be reflected in the survival metrics.
Every statement here helps the team get aligned. A lot of what you are surfacing here are questions other disciplines have about the other people on their team. Think about it: As a PdM, you have a perch above the organization that grants you a higher-level perspective. Other teams, who often work heads down, don’t have your context. When pivots happen without survival metrics, these folks might create stories to fill any gaps in their knowledge. These stories compound and can leave a team pointing fingers. Survival metrics prevent finger pointing.
Again, survival metrics allow for decisions that are fast, politically safe and data-driven.
- Fast: Pivots can happen quickly since you’ve already done this work before the project starts.
- Politically safe: Various team members have less ability and incentive to make up their own narratives.
- Data-driven: By putting all the data in front of everyone on the project, you’re ensuring that you’re not making any decisions out of the blue.
Deploying Survival Metrics
As a product manager, part of your job is taking a ton of information and making it digestible for other departments to help them make more intelligent decisions. To do that, you’ll need to translate the data you’ve found, prioritize them and then set up a cadence to revisit them often. If you need a specific format for your document, take a look at this initiative one-sheet as a way to wrap the survival metrics with great contextual information.
Translation
As we saw above, you translate the data you gathered in your investigations into survival metrics. For the quantitative ones, set limits, like the 10 percent AWS budget in our example.
For qualitative statements, making them binary classifiers is helpful. If a qualitative statement leaves someone unable to decide because it has too many choices, then it’s useless. If you have some researchers on your team, they’re likely a great resource to help translate qualitative statements into metrics. They deal with qualitative statements and research often and can help bring some clarity.
We saw above that the company didn’t want to imitate the Intercom chat feature. We can translate that into a qualitative metric that says if users mention similarities during testing, we need to pivot.
Prioritize
You’ll now have to prioritize all of the metrics against your team’s strategy and company mission. The truth is that some metrics will be more important than others, and those that aren’t important will have to be abandoned. At some point, we’re measuring too many things to be effective; five to seven metrics is the sweet spot. You’ll need to determine which metrics are most likely to make any changes fast, politically safe and data-informed.
Naturally, you’ll use the signals you’re getting from the organization. If these signals are weak (meaning anecdotal or not supported by many signs), feel free to ignore them. If they’re strong (tied to strategy, bolstered by a ton of evidence), elevate them. This will get easier to evaluate the more you use survival metrics.
The company’s mission statement and product vision are a great place to start your work. Including anything you find there in your metrics is important to clarify the importance of both statements to everyone in the organization and help move them from abstract to operational.
Returning to our example, since we saw that Bobco really doesn’t like stealing data from its customers, we must have that value reflected in the chat app’s survival metrics.
Let’s also imagine that we find that the beta population metric you found from the CEO isn’t really important at all. Perhaps you find that most of the sign-ups for the beta weren’t from genuinely interested users. If so, it’s worth removing that INVEST statement before you present it to the team.
Cadence
Now that you’ve translated the statements into metrics and prioritized them, you’ll need to make sure that you communicate these statements to the team often and tie them to what’s happening during product development. That’s why checking in every two sprints and tying those metrics to a retrospective is a good way to see if something is still functioning as a good metric. Keeping the survival metrics fresh allows the team to see their usefulness and make sure you aren’t losing the feasibility of the initiative as a whole.
Now, that deployment is a little clearer, I want to make sure we don’t mistake this for another popular concept: success metrics.
Survival Isn’t Success
We could think of survival metrics as a riff on success metrics, but this wouldn’t be right. Success metrics alone don’t let you know when you need to adjust.
Success metrics are a common evaluative framework for product development teams, and they’re a great way to measure outcomes. Survival metrics, however, are focused on the product’s development rather than its outcome.
- Survival metrics ask, “Is this worth continuing?”
- Success metrics ask, “Was this successful?”
Survival metrics have to be differentiated from success metrics because a project may have been worth it even if it isn’t “successful” when you get to the end. For instance, Steph Curry doesn’t make every shot, but that doesn’t mean he needs to totally revamp his shooting form. His process still has value even if it’s not successful 100 percent of the time.
Both success and survival metrics are important because they measure different things. Success metrics are customer-focused and think about end-state. Survival metrics are far more focused on the team itself, gauging viability as the project is in flight.
So Don’t Forget Survival Metrics
As teams use them more, survival metrics get stronger in two ways. First, they get stronger because each time the team works on a new product, they have more confidence pivoting when the project doesn’t make sense. Secondly, teams can build on the insights of the other teams across an org through reading other survival metrics. This becomes a virtuous cycle by which every department makes stronger metrics and improves their ability to respond to change in real-time.
Given the landmines that often exist, changing projects isn’t just a switch you can flip. When using survival metrics, though, we can face the reality of institutional opposition to change and move forward with the knowledge that we can, as a team, make necessary decisions about a product’s future with the wind of the organization at our back.
