The U.S. Air Force, London’s Heathrow Airport, Kmart and Lidl have all found themselves the not-so-proud owners of large software projects that grabbed headlines — and not the good kind.
The Air Force debuted a new jet, the F25, with close to 900 software bugs, which could cost from $12 billion to $40 billion to fix. Heathrow rolled out a renovation to one of its terminals, only to have a software system failure throw the airport into chaos. Kmart declared bankruptcy after starting and tossing a few supply-chain software projects. And Lidl gave up on a new merchandise-management system after spending about $600 million on its development.
What was the common thread that caused these projects to fail?
The answer is obvious, agilist Vasco Duarte told me. They were “projects.”
A project is defined by its roadmap, or the plan laying out its timelines and components. The corporate world relies on projects to communicate up and down the ladder about what’s next for the company, how long it’s going to take and how much it’s going to cost.
Of course, those last two things are tough to predict. Software makers have to estimate, and those estimates became the center of a hot debate among agilists back in 2012 when agile coach Woody Zuill, along with Duarte and consultant Neil Killick, started tweeting with the hashtag #NoEstimates.
The problem with estimation — and, more broadly, with projects — Duarte argues, is that they’re an outdated holdover from a world that relied on predictive models rather than empirical ones. Now, we can adjust each software development sprint based on the data we gathered from the last one. But projects and estimates lock in a plan of action before that data is available.
What Is #NoEstimates?
What’s the Debate?
Zuill and Duarte marked this 2012 tweet from Zuill as the jumping-off point of the #NoEstimates movement, but both developers, along with agilists like Killick and Ron Jeffries, had been thinking and talking about the problems with estimates long before.
“I tried [tweeting with] #EstimationIsWaste, and there were no takers,” Duarte said. “Woody published one tweet with #NoEstimates, and boom.”
The bluntness of #NoEstimates turned heads — and ruffled feathers.
“What I see happening is the observation that planning is flawed, so let’s not plan or estimate. This is the, ‘Doctor, doctor it hurts when I do this,’ ‘OK, don’t do that,’” project management expert Glen Alleman wrote in a reply on his Herding Cats management blog. “This, of course, is nonsense — learn to estimate and forecast in the presence of uncertainty. Continually improve your ability to estimate and forecast.”
Agilist Alistair Cockburn was pithier: “#NoEstimates is a hoax. Stop it,” he tweeted.
Tech consultant Peter Kretzman, meanwhile, accused #NoEstimates proponents of having “no compelling case” for their ideas, evidenced by what he saw as a tendency to tell people to “just try it” instead of presenting a clear-cut rationale.
Zuill and Duarte do have a rationale for rethinking the use of software estimation. Whether or not software-makers find it compelling is another question. “Almost every company” still uses estimates, Zuill told me.
“The desire to have estimates to give us useful information overwhelms our ability to recognize that we never get that information,” he said.
The Rationale
Here’s a tale as old as time:
Company leaders identify a problem and decide to solve it with software. Project managers scope the problem and create a project plan that includes an estimated timeline and budget. Developers start working on the project and find that the problem requires different solutions than the ones initially proposed.
In that case, developers have two options, Duarte said: Change the estimate, or try to squeeze an entirely new backlog into the existing timeline. In both cases, the estimate didn’t help. In the worst cases, developers enter a “death march” to finish all the new work and still hit the original deadline, which contributes to widespread burnout in the tech industry.
And there are other problems too. That original estimate was probably based on a managerial pipe dream, rather than an honest, on-the-ground assessment by developers.
“If a manager does not like the estimate of a team member, what will the team member do? Stomp their foot and say, ‘I’m not changing my estimate?’” Duarte said. “What typically happens is that you end up taking whatever the manager says is a good estimate.”
“What typically happens is that you end up taking whatever the manager says is a good estimate.”
Even according to estimation advocates, those guesses are wrong about 75 percent of the time. In most other settings, we wouldn’t like those odds.
Releasing our attachment to inaccurate estimates requires a paradigm shift: Companies must view software builds as investments, not projects, Duarte said.
Part of that perspective shift means only developing software when it offers a large return on the cost of building it — 10x, Duarte said. Otherwise, find an analog solution or pay for existing software. It also means analyzing ROI along the way, long before the build is done.
When Zuill was teaching himself to code at night and running a graphics and signage company during the day in the 1980s, he settled into a rhythm similar to today’s CI/CD. He’d build a small piece of functionality at night, then use it immediately at work the next day. If it saved him time, he’d know it was worth keeping.
Those years — along with the writing of thinkers like Peter Norton and Mary Poppendieck — taught him the value of what many call agile development: Building in small chunks and adjusting each sprint based on learnings from the previous one.
“The bottom line of continuous deployment is this,” Zuill said: “What you discover today will change what you really need to do for tomorrow.”
#NoEstimates in Practice
In his very first #NoEstimates blog post, Zuill relayed a story of a project at his prior employer that came with a long and complicated list of requirements. Zuill’s boss wanted an estimate of how long it would take to build all the requirements. Instead, Zuill asked if they could deliver the first requirement and then reassess.
His team built and deployed just one feature out of the many on the list. Immediately, his boss and the users came back with feedback on the feature, as well as an entirely revamped list of requirements. Putting a tiny chunk of the software into production revealed new information that changed the course of the project as a whole. Requirements continued to emerge from the work, and not the other way around.
After doing about a quarter of the work outlined in the original requirements doc, the users had everything they needed to reach their goals, and the project was declared finished. Zuill and his team had never provided an estimate.
That’s all well and good, critics may say, but most companies won’t allocate the budget for software work unless it’s packaged as a project and comes with an effort estimate. Refusing to provide rough estimates is rude to other departments that depend on that software, so why can’t developers just work on making their estimates more accurate?
Zuill acknowledged these tensions — and his tendency is to shy away from broad, controversial statements like “estimates are unequivocally bad.”
“As long as we’re in the complex space, we can’t know ahead of time. It’s like asking, ‘How long is it going to take you to get a cure for cancer?’”
What matters, he said, is the context. He referenced Cynefin, a decision-making framework that groups situations into four “domains” — simple, complicated, complex and chaotic — based on how much we know about them and how clear the relationships are between causes and effects.
Printmaking, for instance, lives in the simple domain, Zuill said. Most every factor is known, and causes and effects involve little mystery.
But software exists entirely in the complex domain. Not only are some factors unknown — often, we don’t even know what we don’t know. Cause and effect, meanwhile, only become clear after the fact — if at all. For example, a team of developers could spend days hunting for a broken system’s point of failure.
“As long as we’re in the complex space, we can’t know ahead of time,” Zuill said. “It’s like asking, ‘How long is it going to take you to get a cure for cancer?’”
That’s why software effort estimation is bad practice, according to the #NoEstimates camp. We know our estimates are based on wishful thinking and that they don’t account for the information we discover throughout the project — yet we continue to make them.
What If I Have to Estimate?
Plenty of developers don’t have the option to politely decline when asked for estimates. Here are some middle-path alternatives:
Story Slicing
Duarte, Zuill and Killick teach the value of story slicing, or taking user stories and breaking them down into even smaller, deployable chunks. In this blog post, Killick encourages readers to approach existing user stories with the question, “What are some options for delivering value to a customer as soon as possible?”
This framing helps developers quickly deliver features designed to benefit customers, rather than projects or programs designed to benefit the company, Killick wrote.
“Either you manage the scope, or the scope manages you.”
Each story chunk should take about a half day to work through, Duarte said. If a story is alive for more than two and half days, there’s something wrong with it, and it needs to be sliced further.
“Either you manage the scope, or the scope manages you,” he added.
Managing Backlogs
When new information emerges, the industry landscape shifts or a project changes direction, that makes your backlog obsolete, so get rid of it. At some of his conference talks, Duarte said there have been audience members with items in their backlogs that were seven years old.
“This is why I argue we should not have backlogs that go beyond two or three sprints because that represents an early commitment to an unknown future.”
“This is why I argue we should not have backlogs that go beyond two or three sprints because that represents an early commitment to an unknown future,” Duarte said. “It’s the opposite of [the agile value of] responding to change over following a plan.”
Measuring Impact
Whether software shops have implemented continuous deployment or are still stuck in waterfall-style project planning, they need to deploy features as they are built and measure impact along the way. As in Zuill’s #NoEstimates case study, the impact of early, prioritized features could change the scope of the entire build. Better to deploy one or two high-impact features than complete an entire list of low-impact ones.
“[In project-based development with estimates,] we are focusing on maximizing the amount of things done rather than maximizing the positive impact of what we’re doing,” Duarte said. “The only way to maximize impact is to learn as you go. There’s no other way, because we don’t know the future.”
The Future of #NoEstimates
While the #NoEstimates hashtag has been around for nine years, the matter is hardly settled. Duarte said he expects to be talking about the topic for a long time.
“I have not seen any indication that the relevance and importance of these ideas will decrease in the near future,” he said. “In fact, the opposite. #NoBacklogs, which is just a tongue-in-cheek call out that our backlogs are way too big, is just another example of what #NoEstimates means in practice. We are committing way too much to trying to freeze the future and set up plans that will not change.”
“I have not seen any indication that the relevance and importance of these ideas will decrease in the near future.”
While this debate has largely played out on Twitter and software blogs, its implications are more far-reaching. According to Duarte and Zuill, arguments like this touch on deeper questions, like, “How long will we cling to project-based management strategies that popped up during the Industrial Revolution?” and “What is the purpose of software?”
According to Duarte, the purpose of software is to create high-return solutions for a small subset of problems. If we continue to build software recklessly to meet upfront project requirements — without considering returns along the way — we put society at risk. For instance, should a government spend billions on a low-return healthcare records software update when it could use the same money to hire more doctors and nurses?
“#NoEstimates is just an example of an agile change that is really a social change,” Duarte added. “It is a change of how we look at work in the IT world. It’s a change of what our priorities are. And it’s a change that will ultimately affect our civilization. [This conversation] isn’t any less important than it was in the early 2000s.”
