“Sometimes you see a very ‘senior-level engineer’ get a company into an incredible bind, with a fortress of spaghetti that only they could ever navigate through,” said Travis Vander Hoop, principal software developer at agile software consultancy Stride Consulting. “You just don’t have that with [extreme programming] teams because everything is constantly being vetted and shaped by the team as a whole.”
Stride uses the extreme programming — also known as XP — method of software development, which incorporates practices like test-driven development, small code releases and pair programming.
Extreme programming has five core values, Aaron Foster Breilyn, a lead consulting software developer at Stride, said. And extreme programming development practices are themselves products of those core values: communication, simplicity, courage, feedback and respect.
“Pair programming helps you communicate with your team, it gives you the constant feedback because you have someone else there to bounce ideas and point out errors,” Breilyn said. “Then when you’re [doing test-driven development], it gives you really, really quick feedback. It helps with communication because you can read all the tests and know what your software is doing.”
What Is Extreme Programming?
Before extreme programming, engineers mostly followed the waterfall method of software development. All the planning for a project happened upfront, followed by continuous development by programmers working on different parts of the project alone, which could last for a year or even longer.
The waterfall method was intuitive, but it also had some big blind spots. For one thing, the agreed-upon project plans often weren’t accurate enough to deliver the product on time. And whenever that happened, customers became frustrated and sometimes received products that weren’t what they wanted.
Extreme programming was created in 1999 by engineer Kent Beck, focused around the idea of getting constant feedback from customers and allowing developers to use their expertise to drive the direction of projects. Beck saw the need to upend the entire software development process in order to prevent those common problems.
While some extreme programming methods can seem counterintuitive at first, many morphed into common practices and continue to affect software development to this day.
Small Releases Allow Developers to Focus on the Minimum Viable Product
When teams followed the waterfall method of software development, code releases pretty much only happened after all the programming was finished and the project was complete. Teams usually decided on a release date at the beginning of a project, set a year or even multiple years in the future.
But there are several problems associated with that way of releasing code. For one thing, it is very difficult to predict the right release date from a year away. Sometimes this causes projects to be completed early and creates excessive downtime for the team before another project is lined up to work on.
More often, however, the time required to complete a project is underestimated. That results in teams blowing through project deadlines, leading to customer frustration and dissatisfaction. It can even affect the quality of the software product: As deadlines tick closer, developers are increasingly under pressure to finish all the requirements, causing them to cut important parts of the software development process, like security checks and thorough testing.
Another problem with the waterfall method is that sometimes, even when a finished product meets all agreed-upon requirements, it’s still not what the customers wanted. That’s one of the worst-case scenarios because, by that point, so much time and so many resources have been invested in a product that may be all wrong.
“A key philosophy of extreme programming is that complexity is the biggest enemy of scaling.”
“A lot of times that is the biggest problem,” said Mukund Srinivasan, managing solutions architect at Capgemini, a digital transformation consulting company. “The team has an understanding of the nitty-gritty [details], but it may not align to what the customer is expecting.”
That can happen when all the technical aspects of a project are correct, but the structure of the project is not built in a way that supports the customer’s needs and vision. Sometimes what the customer is envisioning simply isn’t communicated properly to the team. An effective way to prevent these miscommunications is by having smaller and more frequent code releases so the customer can periodically check whether a project is proceeding in the way they expect.
Developers regularly publish their work to customers, who interact with it and give feedback to the team. That has the added benefit of allowing developers to concentrate on one part of the application at a time.
Small releases allow for more frequent releases, which can happen once every two weeks or as often as multiple times a day. Frequent releases make the process of development easier because mistakes are easier to fix when they are caught early. This is due to the cumulative nature of code — new code builds on top of old code and, as a result, fixing bugs can sometimes require developers to unravel all the subsequent work until the erroneous code is reached and fixed. A single, large release makes the debugging process much more difficult.
“A key philosophy of extreme programming is that complexity is the biggest enemy of scaling,” Srinivasan said. If developers release code a little at a time, it’s much easier to fix any mistakes.
Requirements Should Be Set In Stone
All projects first require an understanding of customer needs. Usually, time is set aside specifically for requirements gathering, the process of speaking to customers to understand their vision for the project and their pain points. Then, in a planning document, both sides agree on the features the application will contain in order to be considered complete.
One of the most innovative aspects of extreme programming is the idea that those requirements from customers should be fluid rather than set in stone.
At first, fluid requirements can sound intimidating. How can developers write software if there isn’t a set list of criteria to write against? In extreme programming, customer needs are still very much driving the project. The trick is to strike the right balance: letting customers set high-level criteria while leaving enough room for developers to make low-level technical decisions.
This method allows developers to rely on their own expertise to think of ways to meet customer needs. Often, they are able to come up with better technical plans. If everything is predetermined by a requirements contract, there isn’t any flexibility for developers to provide more elegant and better solutions.
Fluid requirements also have the added benefit of being more efficient. Project planning takes time, Srinivasan said, and if unnecessary steps can be cut without affecting the quality of the project, that time can be better spent on actually building out the products.
Planning for large projects is usually broken down into several levels. Epics have the most high-level requirements, features are slightly more granular and stories describe the details within the features. Going through all those details with customers can take a long time. While it’s always important to discuss epics with customers, features and stories don’t always need to be hashed out as well, Srinivasan said.
“The customer definition of the high-level outcome, not the detailed-level requirements, provides just enough clarity for the team to deliver the solution,” he said.
Using Tests to Drive Software Development
Even so, not having fixed requirements can come with its own set of problems.
Teams and customers will sometimes misjudge the amount of detail needed during requirements gathering and not be granular enough, even by extreme programming standards. For example, customers might assume an important feature they want is an easy, low-level requirement — only for developers to find out about it later and realize it significantly affects the scope of the project.
One way teams can get around this problem is by relying on test-driven development and “spikes” to home in on what customers actually want. A spike is a rough implementation of a feature that’s usually temporary.
Teams use spikes to quickly test different ways of implementing features. Its primary purpose is to test how an idea works in practice, Srinivasan said. Extreme programming uses spikes to check whether different ideas make sense, and to show the customer something concrete so they can provide specific feedback about what they like and dislike.
“This enables us to be able to create a solution rapidly without having all the requirements, define what the various stages of hitting that very quickly are and build that out in a couple of sprints,” Srinivasan said.
Test-driven development is the practice of writing tests before writing the code those tests target. Though it sounds counterintuitive, it forces developers to think deeply about design and the intention of the code, which helps them write better code more efficiently.
Test-driven development is one way to reconcile the high-level guidance customers give with the lower-level details that development work requires, Srinivasan said. Those smaller unit tests can fill out the broader requirements of the customer-defined requirements, acting as a bridge between epics and story-level requirements.
Pair Programming Improves the Codebase — and Developers Too
Pair programming is an important part of extreme programming, but it’s one of the less widely-practiced elements. While development work usually happens alone — except for the design process and code reviews — pair programming changes development into a highly collaborative process.
Pair programming can be controversial. Detractors argue that having two people work on the same piece of code simultaneously wastes time, halving the amount of work two developers could normally do. But others argue that developers working together are able to evaluate technical issues better and catch each other’s mistakes. If pair programmers are able to produce higher quality code, that makes the development process more efficient.
“Often, beginners ask really insightful questions ... It’s really good for people who haven’t had their designs poked at or questioned.”
Stride relies heavily on pair programming. Most of the production code is written by pairs rather than solo developers and even highly experienced developers can learn a lot from thinking through problems with an early career developer.
“Often, beginners ask really insightful questions,” Vander Hoop said. “It’s really good for people who haven’t had their designs poked at or questioned.”
The practice is also great for building strong engineering teams because one of the most effective ways to teach new developers about an unfamiliar codebase is by pairing them up with experienced developers and letting them dive into the code.
“There are a lot of situations where you have new people coming in and joining [in the middle of a project],” Srinivasan said. “Pair programming enables new development resources to rapidly ramp up and work alongside an experienced programmer.”
Having On-Site Customers Is Not a Hard and Fast Rule
Extreme programming also encourages the practice of regularly inviting customers on site to advise the development team. The idea was to shorten the feedback loop between customers and developers by enabling more interactions. If customers are on hand to answer questions, they can quickly lead developers away from undesirable implementations before teams even have time to code them.
That’s one aspect of extreme programming that’s not as relevant now. Customer feedback is definitely always important, but whether customers need to be on site and how much time they should be spending with the development team are considerations that depend on the team’s unique situation.
“A fairly mature team, where you have strong leads and strong visualization experts, for example, there can be substantially less interaction,” Srinivasan said. “When you have a newer team, you need more face time with the customer, maybe even a couple of times a week, so that the team is aligning to the strategic goals correctly and questions are not delayed.”
The on-site aspect of these interactions is not very important. Adjustments can be made to grow a close relationship between customers and the development team without being in person, like inviting customers to attend daily or weekly scrums where developers share details about the progress of the application, Srinivasan said.
Extreme Programming Has Become Mainstream
Although extreme programming was once considered pretty extreme, these days, plenty of common software development techniques follow its methods.
“When you look at the holistic vision of what Kent Beck was providing — in terms of continuous integration, behavior-driven development, collective ownership — a lot of that is mainstream development,” Srinivasan said.
Short sprints capped by demos of newly released features for customers are now a software development best practice and automated DevOps processes are only becoming more important, highlighting the advantages of small-code releases.
Companies have also adopted new processes that add to these advantages. For example, some teams use feature toggles to release code to production when the timing is right for the business, rather than when features have just finished development, Srinivasan said. This can be helpful if teams wish to hold off on releases during periods when their website or application is heavily trafficked.
While practices like pair programming and test-driven development are not yet universally accepted, extreme programming has permeated most areas of software development. Extreme programming opened people’s eyes to the fact that processes centered around taking advantage of developers’ creativity and knowledge produce good results. When developers are given the flexibility to use their expertise to drive projects, teams may find that some of these “extreme” practices are not quite so extreme after all.
