Product managers (PdM) oversee more than features. Simply focusing on them is a mistake and will leave the product person in charge of nothing more than a feature factory. Feature factories turn PdMs into project managers, which means doing far less research about why we’re making what we make and more simple delivery. Although PdMs can do project management, that’s a poor use of our skills.
Let’s see if this sounds familiar. A stakeholder says they know customers need a widget for a huge enterprise deal. They say to just trust them, they’ve talked to the customer and know what they need. We don't have time to research, and a competitor is already working on this. Just get in there and build.
Take the requirements, build what they say, and produce features. Many teams take those orders and follow through on them. It’s easy to do, and no one gets in trouble.
Well, nobody except the company. The team builds plans that satisfy stakeholders’ fears instead of meeting the actual customers’ needs.
Here is the truth: Companies that do this are executing a terrible version of agile. Worse, they generally don’t understand they’re dismissing the Agile Manifesto. They probably also paid a lot of money to consultants to “transform” their product development processes. Either that or the company is a startup, arrogant and drunk off of the ability to fundraise instead of delivering to the customers.
Either way, instead of operating based on the Agile Manifesto, which we know works, the product development team turns agile into shorter forms of a waterfall development model, a more risk-averse form of linear product development, with none of the benefits.
The manifesto, which built much of the architecture that holds up the top 10 companies of the S&P 500, has turned into a shadow of itself. Likewise, the team’s “transformation” turns into anything but.
Agile Development and Feedback Loops
Responding to Change
Let’s focus on the manifesto’s fourth tenet, responding to change over following a plan. The ability to respond to change instead of blindly following a plan **cough** generally in the form of a roadmap **cough** is the difference between teams that just say they’re agile and those that truly are.
Fortunately, I’ve already given you a tool to help here: Survival metrics help teams focus on the fourth tenet by assisting a product team in determining if an initiative is worth investing in more, pivoting, or stopping altogether. At their best, these metrics provide a shock to the system that forces the team to think about what adjustment to make. This jolt can get teams out of positive feedback loops in which they continually make the same products based on inertia.
But here’s the rub. If you only define survival metrics once and never update them, then you aren’t adjusting to change. Instead, survival metrics become just another plan to blindly follow.
Survival metrics are about change. Change doesn’t just happen at the beginning of a project, either. It’s a constant factor in agile product development. So, you have to adjust your survival metrics to keep up with your development. If you don’t, you’ll end up doing a version of the small waterfall model. Your handoffs will be smaller with none of the benefits that agile brings.
The easiest way to avoid blindly following a plan is to create a negative feedback loop. We can use a tool we’ve talked about before, premortems, alongside another PdM best practice, retrospectives, to create the breaks that allow us to adjust our survival metrics as time goes on.
How do you do that? Well, let’s walk through the process. We’ll follow Bob and Rachel, two PdM's at WidgetCo, over a two week sprint.
Before we do, though, let’s go negative.
Going Negative
Would it surprise you to learn that agile is built on negative loops, not positive ones? The difference between the two is simple. Positive loops offer teams no way to adjust since they are designed to accelerate, while negative loops have a break (think of this as an adjustment period) that helps you get closer to reality.
What does that mean for agile development, you ask?
Well, I would argue most teams live in positive loops. When have you seen your team change direction? The Agile Manifesto tells us to respond to change over following a plan, but ask yourself how often do you find yourself just following a roadmap you built a year ago?
Negative loops happen when something stops the loop, allowing teams to adjust. Think of the negative loop as one where you must respond to something.
Feedback Loops in Practice
Let’s look at this distinction through the lens of two teams. Rachel’s team naturally goes with the flow and, as a result, works in positive loops. Bob’s team, on the other hand, works in negative loops so they can adjust when things change.
Rachel’s team has a shiny, new roadmap. They blindly trust the plan, and after developing survival metrics, they focus on just getting the next feature out. They decide that retros are a waste of time that just slow them down. Instead of doing one every two weeks, they just continue to build whatever the plan tells them to as long as no one is visibly angry.
Bob’s team also has a shiny, new roadmap. They trust the plan but verify it before beginning. They run a premortem and create survival metrics using the data they get from the workshop. They work with the metrics for two weeks. After that time, they take time to do the retro they have planned. When they look at their work, they find something they didn’t expect.
Using the Premortem
The premortem is a workshop where the team gets together to envision what could possibly go wrong during a project and adjust the plan to account for those eventualities. This may sound like ordinary planning, but what makes premortems different is that the workshop format allows a mixture of voices that can lead to new insights.
A neutral observer needs to facilitate these workshops. This facilitator needs to move the conversation forward and keep notes. Without a facilitator, this workshop can go off the rails.
Without further ado, here is a template for the workshop:
-
Check-in. Spend a few minutes trying to understand everyone’s mental state by asking, “How are you feeling?”
-
State the objective. Ask everyone to write the objective of the product down on a piece of paper, then have the facilitator check all the sheets. If the objectives match, hooray! That means everyone has done the reading. If they haven’t, then spend the next few minutes aligning the team around the goal.
-
The project has failed. Tell the participants to imagine that the project has failed, and ask them to write one or two reasons why. Do this as a brainwriting exercise. You want everyone's unfiltered point of view.
-
Vote. Which problems seem the most likely? Pick the top three.
-
Discuss. Talk about the “why” for each of these problems amongst each other and come up with a few action items to avoid the impending disaster.
-
Build. The facilitator works with the team to turn the action items into something the team can track. This output can either be eigenquestions if they are going to guide the product delivery process or new survival metrics if there is a trackable change in the process.
-
Vote again. Hold another vote to see if this new set should unseat the other survival metrics the team started with. Remember you only want five to seven survival metrics in total. Ask whether these new metrics or the old ones will move in the next few weeks.
Rachel’s team doesn't take this process seriously and skips it, preferring to follow the roadmap.
Bob’s team does this workshop and finds two of the original survival metrics aren’t going to move the needle. They initially thought they had to adjust their development based on the proposed marketing budget. After an infusion of capital and the hiring of a new head of marketing, however, they realized they had much more leeway.
They replaced the marketing metric with a survival metric focused on team connection by assessing how aligned the team is on the overall objective. If Bob felt alignment was off, then the team would need to adjust their communication plan. They also find that operations team is nervous about the AWS bill, so they added that to the list of survival metrics. They also found out the infrastructure bill was an issue after some development, and put it on the list as well.
Two weeks later, the retro provides the perfect time to do a survival metric retrospective on the last sprint.
Survival Metrics Retrospective
So, in the retro, we analyze what has happened, and we probably have to adjust to a new reality.
Retrospective exercises help your team make these adjustments by making reflection a regular part of the team’s routine. Instead of going with the plan, they have the opportunity to stop themselves and change by cobbling together action items. Because of that, the team gets to adjust over time.
You want to answer several critical questions with a survival metric retrospective, but the most important is this: “Are these metrics affecting our product development over time?”
Essentially, survival metrics are useless if they aren’t touching what really happens with product development. They are active metrics, designed to help change direction when things are happening. If you aren’t feeling the burn of the survival metrics, it is time to change direction. The workshop will help you to see how well they’re working.
The workshop works as follows:
-
Check-in. Just like before, spend a few minutes getting everyone's mental state coordinated and understood by asking, “How are you feeling?” Retros are a bit tougher than premortems since they generally come with more trepidation because we’re looking at work we’ve actually done instead of hypothetical situations.
-
State the objective. You may ask why you need to do this again. Well, this is also a chance to make sure that the team understands what they are trying to do. It also gives you, the PdM, an opportunity to adjust your communication strategy. For the PdM reading this, any opportunity to repeat objectives is a good one because objectives have a short lifespan in a team’s mind.
-
Consider. “How have these metrics affected our product development over time?” Go through the metrics one by one and ask how they affect development. If nothing is happening, get rid of it. Time for new metrics.
-
Ask. What is actually affecting our product development? These answers are potential new metrics.
-
Adjust. The facilitator works with the team to adjust. Sometimes there isn’t anything to change, but most of the time, there is. The important thing is to walk through your current metrics to see if they reflect reality.
Rachel’s team doesn’t do this type of evaluation and keeps the same metrics. As a result, the exercise becomes another useless concept that takes the team’s time for no reward. After two initiatives, survival metrics go into the dustbin.
After a few more cycles, Rachel and the team look at the feature’s usage numbers with shock. The numbers were nowhere near where they expected. The initiative also came in over budget. They didn’t adjust, and the team ended up losing long-term trust as Rachel had to explain the poor usage numbers.
Bob’s team takes this opportunity to change three more metrics. The team has a good hold on what’s useful and eventually stops the initiative. They saw the writing was on the wall. Since they were able to go through the retro process, they were able to speak to other parts of the business about why what they were building wasn’t effective. The team finds that what they have built will blow the budget out of the water, and since they were a cost-conscious company, raising this fact to leadership kept them from building something that wouldn’t be useful.
Over time, Bob’s team gained the trust of the whole organization. Bob eventually got promoted since he became known as the PdM that shipped all impact, no filler.
Don’t Fear Negative Loops
The negative loop is important because we have to respond.
Responding to change keeps strategy alive. Remember, as PdMs, our job is to raise the decision quality over time. We can’t do that if we aren't adjusting to change in front of us. Leveraging negative feedback loops through premortems and survival metric retros from time to time take you away from the humdrum of following a plan to get closer to the dynamics of what is really happening.
