“We observe that some methods still considered standard for developing software have long been abandoned by other disciplines.”
That’s a line from the introduction of Lean Software Development: An Agile Toolkit, the book Mary Poppendieck and her husband, Tom, wrote in 2003.
At the time, software development was suffering from a mistaken belief: that building things fast and building things well were fundamentally opposed. Mary knew this to be untrue from her work in product development and manufacturing, where the “lean” practices that sprung up in Japanese car factories and subsequently spread across the globe often ruled the day.
Software had fallen behind, Mary and Tom wrote 17 years ago, and an emerging set of principles called agile — along with some thinking from lean manufacturing — could set things straighter.
Today, Mary puts it differently. Agile, though not bad in its original form and intent, is irrelevant, she told me. Her reasoning may sound familiar to anyone who read Lean Software Development: Other disciplines are leaving it in the dust.
“You’d never hear anyone say, ‘We help mechanical engineers be agile,’” she said on a Zoom call from her Minnesota living room. “That would be silly. And I mean that in the worst possible sense of the word.”
It’s true — agile engineering has been around for a long time, in tech industry terms. But instead of becoming an AltaVista, agile became an Amazon. Nineteen years after the Agile Manifesto was formalized, agile is still on the tips of tongues, from everyday developers to corporate executives.
Mary Poppendieck, however, is ready for the next thing.
What gave rise to Agile?
Software development organizations, according to a popular essay from Mary’s blog, are either cost centers or profit centers. A business’ goal for a cost center is to keep spending down. Its goal for a profit center is to maximize revenue.
Most companies pre-digitalization operated software cost centers in the form of IT departments. That meant software wasn’t seen as an integral element of business growth, and the people overseeing software teams often had little understanding of technical work. As more companies caught on to software-as-product, that myopia didn’t go away.
In environments like that, Mary said, agile was a call to cut back on red tape and get more power into the hands of developers. It was primarily a communication tool, clarifying communication with customers through test-driven development and streamlining communication among teams by overhauling Waterfall-style development processes.
“The biggest error in agile came from its roots in the ’90s, and that was that enterprise software became an endeavor of doing what somebody told developers to do.”
The agile engineering community — with its programmers, businesspeople and theorists — championed innovations like unit testing, extreme programming and the waste-cutting lean programming of the Poppendiecks. But, like much else in the world of tech, good things eventually get usurped by something better.
In this case, that something better is a business model. Today’s top tech companies don’t have software departments, they are software departments. Executives can speak technicians’ language, and companies view development teams as key revenue-drivers, not costs to curtail. Yes, customers want functions and features, Mary said, but more than that, they want solutions to their problems.
“The biggest error in agile came from its roots in the ’90s, and that was that enterprise software became an endeavor of doing what somebody told developers to do,” Tom said. “Agile was very good at delivering features, but it didn’t make any difference. Companies were no better off when they had features faster than when they had features slower. The focus needs to be not on features, not on output, but on outcomes and the impact of what employees are doing.”
In other words, Mary said, software developers need to become what they truly are: engineers.
Has software strayed from engineering?
Software programming, in its infancy, was often used to automate jobs that traditionally involved two factors: electrical systems and women to operate them. When Mary worked for Bell Telephone Laboratories (now Bell Labs) in the 1960s, she was programming computers to replace the relay switches in telephone systems across the world. The women who had operated those switchboards were out of jobs — while Mary and two of her sisters launched careers as programmers.
When Mary later worked in process control programming at 3M, her work closely paralleled that of electrical engineers.
“I was in an engineering department. My title was engineer and we worked as engineers,” she said. “What I was doing in electronics, other people were doing in wires. But there wasn’t much difference in the stuff that I did and the stuff that they did, except that computers were an interesting way to do stuff that they used to do with circuits and relays.”
“Way too much of agile has been not about technology, but about people and about managing things and about getting stuff done — not necessarily getting the right stuff done.”
Today, however, many don’t view software development as an engineering endeavor — or, more likely, are unable to because of obstacles in their organizations, Mary said. Engineering is about seeing a problem, understanding it and using the technology you’re good at to solve it as quickly as possible. Software, she said, has become about “projects, briefings and being nice to people.”
“Way too much of agile has been not about technology, but about people and about managing things and about getting stuff done — not necessarily getting the right stuff done, but getting stuff done — rather than what engineering is about,” she said. “Agile has come to mean anything but the fundamental, underlying technical capability necessary to do really good software engineering.”
So why has agile clung to the soft skills that made it so relevant in the befuddled software shops of the 1990s? It may be because hard skills are hard to sell.
How ‘soft skills’ took over Agile
Google “agile coach” or “agile consultant” and you’ll find no shortage of people offering to help your team use agile practices to its benefit — for a price. To be fair, Mary and Tom themselves charge money to teach workshops on lean software development, which is rooted in agile.
But, by removing any mention of technical issues or skills, some brands of agile training widened their potential audiences, Mary said. She named a few reasons this bugs her.
One is that this approach runs the risk of positioning agile as a quick fix for companies still struggling to digitize or clinging to a cost-center view of software.
“They want a process to make sure that software is done right, instead of considering that maybe their fundamental concept of what software is is wrong,” she said. “Find me an actual tech company that talks much about agile, and I will be astounded.”
“Find me an actual tech company that talks much about agile, and I will be astounded.”
Her other objection is that, by focusing on so-called soft skills instead of actual engineering prowess, organizations that subscribe to technically bankrupt forms of agile could end up pigeon-holing women engineers into roles Mary sees as peripheral.
“If you look at agile, where do women end up? They end up being scrum masters and that kind of thing. That’s not an engineering job. That’s putting women ‘in a woman’s place,’ rather than putting women in an engineering job. And I think that’s really bad,” she said.
Her concern isn’t coming from a vacuum. Women dominated computing professions from their inception in the 1940s all the way through the 1960s. As the scope of computers’ usefulness became clearer, however, those jobs started going to men. Women programmers made a comeback in the 1980s, making up 37 percent of computer science graduates in 1984, but their numbers have dropped steadily since then, beginning right when the first personal computers appeared. Today, they make up about a quarter of computer science graduates.
The gap in who learns to program is accompanied by a gap in career outcomes. Women programmers over 35 are 3.5 times more likely to be in junior positions than their male counterparts. When they are promoted, they’re sometimes pushed into non-technical roles that emphasize soft skills. These roles, in many organizations, are seen as less prestigious and less desirable.
According to Mary, if agile methodology creates more non-technical roles and steals focus from technical problem-solving, its continued popularity could work against women developers.
What we can learn from rocket boosters
Agile was a lot better than what software development looked like in the ’90s, Mary said. It had a good run in the 2000s before continuous delivery came along and, in many ways, made agile’s development units outdated.
Now, continuous delivery is what’s expected, and the industry is ready for the next thing. But that next thing shouldn’t be another methodology, according to Mary.
“There is no methodology in my field of software engineering that can conceivably last more than five to eight years,” she said. “Everything that is 10 years old is obsolete. Everything that is 20 years old is archaic.”
Furthermore, she said, methodology requires codification. Beginning with the capability maturity model in the ’90s, software development methodology meant developers had to show they had standards and that they followed them, rather than demonstrating that their standards were constantly in flux depending on consumer needs. That’s the definition of quality standards lean manufacturing practitioners in Japan originally espoused, Mary said, and they’re not compatible with methodology. Instead, they’re all about learning.
To that end, Mary is excited about all the ways artificial intelligence will allow software engineers to learn better and faster. Automated testing, continuous deployment and cross-functional collaboration are now table stakes, Mary and Tom agreed. Cutting-edge companies will discover the next great approach through an engineering mindset and a willingness to learn.
“Today, my favorite word is ‘engineering.’ Just let engineers be engineers.”
Sometimes, that learning could come from advanced AI models like neural networks. Other times, it could come from good, old-fashioned trial and error.
“Have you been to Mars?” Tom asked me.
I haven’t.
At my age, I could probably make it there, he said, and SpaceX would be the one to take me. They’ve blasted to the front of the aerospace industry by employing a classic engineering model, Mary said: Just try it.
During the company’s development of reusable orbital rocket boosters, for example, the boosters crashed again and again and again. Despite what looks like a lot of failures, SpaceX has cut the cost of rocket launches by two thirds, compared to other NASA contractors, largely by trying things that had never been done before.
“They are doing rapid iterations and experiments and instrumenting everything to get as much learning as possible from every investment they make,” Mary said. “They don’t call it agile, but they’re really good at it.”
Being good at things, it seems, is Mary’s current creed. Buzzwords come and go, she repeated, but smart, technical problem-solving will never feel dated.
“I don’t care if it’s lean or agile, there’s no silver bullet where if you just follow this formula that somebody else followed, you’re going to be great,” she said. “So today, my favorite word is ‘engineering.’ Just let engineers be engineers.”
