Quality Assurance Owns Agile Acceptance Criteria

I've been doing the Agile development process for a few years now. I've worked on different size teams and I've worked in different Agile environments (by this I mean the processes and tools have differed from one team to another). I've learned one of the key factors to working efficiently and effectively is to have clear acceptance criteria for the sprint tasks.

As a QA engineer, it is up to me to make sure the acceptance criteria of a task is written with clarity and completeness. In this sense, I believe QA "owns" the acceptance criteria. Of course the product owner, along with the rest of the scrum team, will contribute to the criteria, but the QA person should feel a sense of ownership of the criteria. This article recommends some steps you can take to ensure your sprint tasks have top quality acceptance criteria.

First, a little bit about the Agile process. Typically, the product owner adds tasks to the backlog and prioritizes them. The scrum team gets together on a regular basis to review and flesh out the tasks. It's during the review process that QA needs to be active and involved in driving clarity and completeness for the acceptance criteria. Don't size a task until you are happy with the acceptance criteria.

Clarity
"Eats, shoots & Leaves", by Lynne Truss, is a well-known book about grammar and punctuation. The title intentionally conveys an ambiguous message to point out the confusion poorly written words can deliver. You may not think this grammar and punctuation stuff applies to criteria but if you've ever seen a criterion like "When the user sends an email there's a notification, too" you begin to appreciate and call for good grammar and punctuation. Who gets the notification? The sender? The recipient? Both? What's in the notification?

The author of that acceptance criterion didn't set out to be ambiguous. Most likely the author assumed everyone else on the team knew the notification was to go to the sender and no one else thought to ask. Look for a criterion that calls out more than one user, either explicitly or implicitly (as in the example above). Be sure the criterion is clear which user is associated with what actions. 

If a sprint task calls out certain actions, are those actions the same for every user? Do the actions vary based on the qualifications of the user? A web site form may allow a non-logged in user limited access, while a logged in user has more access and a logged in user with administrative powers can do even more. All these users could see the same form, but the allowed actions vary, and need to be documented in the task's acceptance criteria.

Completeness
It can be tempting for the team to focus on just the happy path for the new feature task. It's important that you think about other possibilities and bring those into the discussion. What happens if the user doesn't complete the web form before submitting it? What happens if a bogus email is used? What happens if the same email is used for two different signups? What are all the ways a user can encounter this new feature? When this user takes an action, who else in the system is impacted?

When my team developed a chat application we had to always keep in mind how an action by one person in the conversation would impact everyone else in the conversation. Some of our initial tasks only talked about the action taken by one person and did not call out if or how others were affected. We got better about doing this as we worked on the product. 

If the task includes API work, you have probably already considered how bad input is handled. What about parallel invocations? Does the API assume things about the back end that may not be true? If the API is part of a sequence of calls what happens if an upstream API call fails?

These are the types of questions you probably think of while coming up with test cases for the task, but they are just as important while the acceptance criteria are being written down. Don't be in such a hurry to assign story points to the task that you forget to carefully consider how well defined your tasks acceptance criteria are.

Beyond the Task
One of the challenges with Agile is to do testing beyond functionality and usability of the specific task. How you handle testing of quality areas such as security, compatibility, interoperability, performance and scalability impacts your acceptance criteria. I've been in organizations where we dedicated a whole scrum team to take on these areas only. This meant other scrum teams could focus on functionality and usability. I've also been in organizations where each scrum team had to take these on. Whatever the size of your organization, it's important your organization decide who does the necessary testing of these areas. 

If your individual scrum teams are responsible for any of these areas then you will want to have a set of questions related those areas. You can start with basic questions and dig deeper, if you feel it necessary. What are our security concerns for this task? What are our performance targets? Does this task need to interoperate with any other application?

Conclusion
In addition to these specific recommendations, it's important you develop a thought process and refine it. Whether you are already doing Agile or just getting started with it, I recommend you have a process you use while you participate in the definition of acceptance criteria for a task. As your team takes a backlog item and fleshes it out, apply your thought process, ask your questions, and review the acceptance criteria. Own the acceptance criteria.

While you go through sprints you may find some tasks' acceptance criteria need adjustments. They may be lacking clarity or completeness. When this happens take it an opportunity to review and refine your thought process. What questions could you have asked up front to avoid this modification? Did you overlook areas or forget about different user types? Analyze and refine.

Another benefit I've noticed is that over time, when I keep using this thought process, the product owners start to anticipate what I'm going to ask and write better acceptance criteria from the get-go. This speeds up the whole team and makes us a leaner, meaner Agile sprint team.