Save Yourself a Headache and Document Your Project’s Acceptance Criteria Early On

When product managers and software developers talk about the “definition of done,” they’re really talking about acceptance criteria. It’s a checklist accompanying each user story that describes when the requirements of the user story are fulfilled.

Customers, developers and other stakeholders on a project reference the acceptance criteria to determine how the final result of the project should look and behave. It works well because every party gets something out of including these parameters in the project.

“For me, it’s definitely about organization and preventing scope creep, which are super common.”

Although customers are not always directly involved in setting acceptance criteria, having those criteria helps the development team focus on the important aspects of user stories, ensuring customers that the development process will be focused and will satisfy their needs.

Product managers, who sometimes run acceptance criteria by customers before finalizing them, are able to use acceptance criteria to show customers that the work they agreed to was satisfactorily completed, and to push back against scope creep.

But for the most part, acceptance criteria are written for developers, who use them as a guide to create the software. They focus developers’ energy on the project requirements and essential functions and prevent them from getting sidetracked with unnecessary features.

We spoke with people experienced in software development, product management and agile methodology to learn how they create acceptance criteria.

How Do Acceptance Criterias Work Within the Agile Framework?

DHAVAL PANCHAL
_{Founder and agile coach at Evolve Agility}

The practice of acceptance criteria comes from extreme programming. What extreme programming tried to do was look at software development and say, “What are some things that we do in software development that works for us?” — and now we are going to turn the dial to 10 on those practices. Extreme programming introduced a technique called user stories because we really want to engage in a conversation with our customers to learn what they think our system can do for them. So let’s have a conversation where the user is going to tell us a story. So it was like a two-way conversation that was happening. Now, as a result of this conversation, they would write the whole requirement on an index card. Those then became the acceptance criteria, which, as an engineer, I would then take and convert them into an automated test case.

JESSICA BAHR
_{Scrum leader at Charles Schwab}

Acceptance criteria allows teams to know when they’re done and that flows into literally everything else. Because how can you plan, how can you estimate, how can you be reliably delivering at a certain pace? All of that relies on the ability to have clearly articulated acceptance criteria.

DIANA RODRIGUEZ
_{Full-stack developer and automation specialist}

Any user story should have at least one acceptance criteria, that’s for sure. It’s not something that you should skip. This should be done before implementation — and a lot of teams actually miss out a lot on that. They write up acceptance criteria after things are implemented, and then you miss out on the benefit from planning things ahead and knowing exactly what the “definition of done” can be.

It helps to manage expectations and to define scopes. The less ambiguous things are, the better. And it’s important to define these things from the very beginning, then it can move from managing expectations to also handling things like QA. We can establish good criteria for the QA side of things.

JEFF MARTENS
_{CEO at Canary Monitoring}

It kind of depends on the type of product it is. So if it’s something that has a large user interface element, you will probably have some conversations with a customer about how they might want to see things work. You might run a design by them. That’s a little bit different than an acceptance criteria, but it’s kind of part of this, “What are we trying to have when this ticket gets done? What do we want to be true?” And in some cases, that can be very design heavy.

TODD WILLIAMS
_{President and consultant at eCameron Inc.}

You’re not going to know what the acceptance criteria is for an entire agile project, because you don’t know what you’re building yet — so you have to write acceptance procedures for each sprint. However, as you start going from sprint to sprint, you’re going to find out you’re going to have to do some sort of regression testing — not just regression testing from a developer standpoint, but from an acceptance standpoint.

Agile wants to be lean on documentation, but trying to test the test cases and making sure something is built robustly — that takes documentation. I don’t know any other way.

More on AgileWhat Is an Agile Coach? How Can They Transform Your Team?

What Are the Benefits of Having Acceptance Criteria?

ERIN RANDALL
_{Agile coach and founder of Ad Meliora Coaching}

What I love about acceptance criteria is it’s a really clear up-down vote: Is that story complete? I don’t need to be second guessing, is this done or is this not? Does it meet the criteria that were laid out here? Yes. Okay. Then you can move it to done — and not until then.

Bahr: As humans, we need to know when we’re done with things. Anytime we are told, “Hey, I need you to do this thing,” the first question that should come up in anyone’s mind is, “Okay, what would make this a good outcome for you?” And it’s the same with teams. It can be very easy for those teams to get into the weeds and then find more problems and more problems and want to just take care of all of the things — and then it becomes death by a thousand paper cuts. And what acceptance criteria does for the team is it says, “Okay, this piece of work is done when you can meet these checkboxes.”

Rodriguez: Everything having a clear definition of “done” protects the client and the team. It’s fundamental for teams to be organized, to use energy and resources in an efficient way.

It protects developers because we tend to go over the scope easily and do a lot of work that is not documented properly, that is not tracked properly. It’s mentally and physically consuming — it takes a toll, you know, to work on certain projects. So that’s the best way to organize and to predict and prevent chaos and to keep a team well greased and working smoothly — it just makes the process more manageable and more transparent as well. If you have acceptance criteria, you can give accurate information based on everything that is agreed.

It’s also a good way to onboard new team members and to organize the way a team works and improves delivery. But for me, it’s definitely about organization and preventing scope creep, which are super common.

Martens: If you focus on outcomes, that can really guide your conversations, your planning and your development a lot. Outcomes are the ultimate thing — if you empower people to know those outcomes, they will be empowered to do the best work they can to reach them.

Who Should Write the Acceptance Criteria?

Randall: Generally, the product owner writes user stories and acceptance criteria. But for the best teams I’ve seen, this is a conversation and not a one-sided conversation. The product owner may ask for a particular feature set, but then he or she also wants to hear from the people that will be building that feature, “What am I not considering here? What have I missed? How could this be better?”

Rodriguez: The client, a product owner, a product manager (PM) and a solutions architect define the acceptance criteria, and then it’s just passed down to the developers, eventually. Some people find it a bit extreme, some people think just a PM and a solutions architect will do and that’s totally fine. I just like to have different perspectives involved.

The solutions architect, based on the feedback they get from the client, they will think of architecture and implementation. The product owner will think of creative ways to implement these features that will enhance what the client wants. The PM will organize these things. And then all together, we do refinements on those tasks, and make sure that deadlines are met, that the team is working appropriately, that we have all the information and all the conditions that are necessary for the project to be carried out properly.

Martens: I typically think it is the job of the PM — or whoever has been tasked with writing the user story. I don’t always think that needs to be the PM. But I do think there needs to be a very clear definition of who’s responsible for this. I don’t think that’s something done well by committee — it is kind of a one-person task. So whoever you’ve tasked with doing that, it’s their call.

I look at product managers as the voice of the customer. That’s part of their job. So it shouldn’t be hard for them to know what the customer wants. I also don’t think that user stories should be written, thrown over the wall and never looked at again. What I like to do is write the user stories and review them with the tech lead or the engineering manager before rolling it out to the rest of the team.

Williams: I want to make sure that people understand this distinction between the acceptance test criteria and who approves that, and the acceptance test run — the person who’s actually made to go through the steps and sign off and say yes, this functions as it should function.

That is sometimes a completely different person. So you may have someone in purchasing or someone who is in a managerial role who approves the actual acceptance criteria. Yet, when you actually test it, it probably isn’t someone in the information technology group. It’s an end user who’s actually going through and using that software.

What Should You Consider While Writing Acceptance Criteria?

Randall: It’s oftentimes done during backlog refinement — that is a hugely important meeting. That’s where you’re really having that conversation between a product owner and the team, the people who are going to be doing that work. It’s for the product owner to understand what he or she is asking the team to do, and the team to also understand the “why” of the work — how is this valuable? Good, clear acceptance criteria make it easy for a team to work together — to get more work done, but the quality is going to be better as well. You can do some in sprint planning as well, that’s when you’re really finalizing what work you’re going to take into a sprint.

Panchal: The practice does not stop at writing the acceptance criteria. Don’t forget to write an automated test case — your acceptance tests that check for something that the user really cares for. What you want in the long term, after six months of development, is that this customer could come and press the acceptance test suite button and see that all of these tests are passing. So whatever new items the team has added does not break the stuff that used to work previously, from the customers viewpoint.

Rodriguez: The criteria should be independently testable. It should be as clear as having a pass or fail. Something simple — the more complex and longer the sentences are, the more complicated it is to achieve that pass or fail.

At the end of the day, the acceptance criteria is not only about protecting the client and the team. It’s to make deliverables understandable, approachable and clear for people who don’t have technical backgrounds — in a way that we can show deliverables and explain the results without having to get too deep in the nitty-gritty complications behind a task. It’s basically the best way to show results. We agreed on these criteria, we met them, they were tested in this particular way.

Martens: There are two schools of thought with product managers and acceptance criteria. Some product managers say, “I don’t want to leave any room for interpretation, I want to say exactly how I expect this product to look, work, everything.” And then there’s another school of thought, which says, “I’m not going to prescribe where every pixel on the screen goes. I’m going to focus on what I want to accomplish, set the boundaries, and then let my team figure out the best way to do it.” 

I personally am in the latter part. At the end of the day, I don’t believe that I know everything. And I just don’t think we build the best software that way. But what does matter is outcomes, right? I think if you focus on outcomes, you’re more likely to get that outcome, because you’ve made it more clear. And secondly, you might get there in a way that is better than what the person writing the user story would have come up with.

Williams: So the developer who thinks in the way of how the code is being written will look at functional specifications a certain way. But the person who’s actually going to be using it looks at it totally from the interface standpoint. Whenever an end user looks at acceptance test scripts, they suddenly say, “Oh, I see how this works. I don’t even need a user manual.”

My favorite example of a user acceptance test was for typing in users’ first names and last names. And the name the person just happened to have on the tip of their tongue was “Maria De La Costa.” And once you got spaces inside that last name, the system started to choke, because it wasn’t expecting that. So those questions came up the minute they started seeing the acceptance test procedure, then they started asking questions, “How is this actually going to work in this situation?”

What’s an Example of Good or Bad Acceptance Criteria?

Panchal: It’s bad when the acceptance criteria is treated as a contract — when what is written on the card is more valuable than the conversation. The negative side effect is that the collaboration between the customer and the developer breaks down. The intent for acceptance criteria and user stories was to break away from the business and IT divide — it was intended to bring the two parties together so they could have a dialogue, learn about each other’s needs and iterate very, very quickly. But if we now place the acceptance criteria as a new contract, then it is going to become challenging for the business and IT people to collaborate again.

Bahr: Bad acceptance criteria read like an outline to a novel. It’s too much. It causes more time consumption just making sure that you’re meeting every single one of those little details. It’s also incredibly difficult to read through and to have a complete picture of what you’re designing. We say in coaching, “Clear is kind.” So if I give you a whole bunch of acceptance criteria, and your first language isn’t English, which is the case for a lot of our developers — holy cow, is it difficult to understand what the piece of functionality is like at its heart.

Rodriguez: A good user story: As a teacher, I want to generate a grade report or some kind of assessment report so I can evaluate student performance. So the acceptance criteria for this could be to show the students’ current scores, display past scores of students, and provide options to share or print this information.

This is quite narrow, but it basically goes straight to the point: We have a user story, we have concrete tasks that need to be carried out that cover possibilities of things failing, and we cover basic operations like saving, sharing, and importing or printing specific information.

A bad example is something like, “Display error messages,” instead of seeing something like, “Show error messages if the service is not responding, or if there’s a timeout.”

Martens: My team just did this today: I just wrote a bug ticket to fix something because I didn’t think we had good acceptance criteria. We said, “You should be able to log in,” but we didn’t consider the outcome of signing up. And in hindsight, what we wanted to do is allow an existing person to log in and not allow an account that doesn’t exist to sign up. But we didn’t talk about that in our acceptance criteria or user story — we just said there should be a link to the sign-in page. But if the story is written, “Put a link there so someone can go in,” you miss all of these other things that are actually important.

Further Reading9 Strategies for Writing More Effective User Stories

What Does the Format for Acceptance Criteria Look Like?

Randall: We often use a format of “given, when, then.” So, in this specific scenario, given this, when it does this, then it does that. I don’t like to make this more difficult than it needs to be. And oftentimes for teams that are new to working in that format, it’s good to be a little bit more strict with following that. But as the team gets more mature, they have their own language with one another a bit.

Panchal: Most people use simple English statements or they will use the “given, when, then” construct. In some situations, it helps to provide an example table. If I am trying to provide clarity on some formulaic work, I might use an example table, or if I’m trying to provide some clarity on rules being followed, there will be an example table used as an acceptance criteria.

Look for compounding English statements — that have the potential to be split into different acceptance criteria. English can get confusing. If you’re using these compound sentences, it can alter the meaning.

Bahr: There is a difference between acceptance criteria that is prescriptive and acceptance criteria that is meant to qualify the work that has been done. Acceptance criteria is about outcomes, about having a clear and concise statement about what you want the outcome to be. If you have a whole bunch of extra explanations, getting to that important outcome takes longer. And you’re trying to make sure, “Wait a minute, what did they mean when they said that?”

How Specific Should Acceptance Criteria Be?

Bahr: My favorite piece about acceptance criteria, honestly, is that it allows the team to be flexible with the “how.” The acceptance criteria says, “This is what you must do,” but it doesn't prescribe the method by which that’s accomplished. As long as they’re checking the boxes, it allows them to be creative. Nobody likes to be told what to do and how to do it. To come in and prescribe the method for solving a problem is very demoralizing, so it’s about keeping people engaged as well.

Rodriguez: We’re trying to focus on the end result, on the “what.” Getting too caught up in the “how” is not an approachable way to present deliverables. I need this to move from A to B. Did it move from A to B? No? Then fail. At the end of the day, they just want to see the results.

The thing is, things that are understandable and have context can help the team know exactly what they need to do — simple and plain English, short sentences that explain very precisely what needs to be done. Be as specific as possible without writing too much.

Martens: I think it really depends on who your team is. There’s no such thing as the best way and the worst way to write acceptance criteria. It really comes down to the culture of the company you’re at, and what are the skills and culture of the team that’s working on it?

Do we trust this team to figure out how to do something and how to meet business objectives? When you do that, you have a team of people who want to input their creativity and their experience into how to accomplish a user story. But then you have some companies where the culture is that your boss will tell you exactly how to do everything.

The best way is to lay out clear business objectives for the company and then let the product manager come up with user and product outcomes that they need. And then let them decide how to do it, and not be too prescriptive. But I think that means you have to have a team that likes to collaborate, and you have to have software engineers who want to be collaborative about the solution and be creative and bring their expertise to the table — not just on technology but on product.

For in-house teams versus outsourced teams, my experience has been that when you outsource software development, you have to have much more detailed and specific acceptance criteria. And it’s typically because those teams don’t have the context that you have — they’re coming in to work on a project, they’re not part of your company, they don’t know the background, they don’t know the business objectives — they just know that the software needs to get developed. There’s not a lot of room for ambiguity. So that’s a very specific example of why you might have to consider a different way to write acceptance criteria.

Responses have been edited and condensed for clarity.