Browse All Articles
> Some reminders about running a UAT (User Acceptance Test)
Recently I and the team has just gone through a series of UAT with the client over the weeks, and would like to drop something about what we learnt during the course. These are not comprehensive bullet points telling how a UAT should be prepared. They are just my own little tips, and having these written for now in case I get to be the person managing it one day.
UAT is not a time for reinventing the wheel, neither is UAT a chance for final prototyping or design review. If there are still requirements not yet settled, that should be a good indication of not starting UAT. In fact it makes a great deal of sense to work out a list of check points with the client, which shows the critical aspects without which being fulfilled UAT cannot commence.
Should that be just a single departmental workflow? Or getting any third-party vendors involved? Or even to an extend, crossing the organization? Could these be split into several rounds instead of one go? A time to understand the situation and to compromise.
Address the list of check points as the base line, so as to remind the team of the targets they need to be shooting for. Do the same to the client management and the testers, that they shouldn't feel any disappointment if there will be nothing more or out of the list.
Last but not least, avoid getting into the situation of "start UAT just for making a start", which is probably due to some unrealistic deadlines set out in the project plan, and someone needs a "tick" in that plan.
UAT as it self-describes, is often interpreted as an assessment to the system. Testers tend to believe in "what you see (now) is what you get".
They hesitate on things that don't work the same as their daily operation. It is no good to say "We just show it here, now can you imagine how it would be like in the future..." because this becomes less persuasive in UAT and really indicates it is not quite ready.
The set-up of the environment (is it mimicking our receptionist counter area?), equipments (is the computer monitor of the same resolution?), devices (is the barcode scanner of the same hand grip?), print-outs (would the sheet/label come out from that printer right after I clicked?), and operating system (internet browser, font, permissions, user login ... ) need to relate back to tester's workplace and familiarity.
Taking care of this friendliness factor and making it as real-life as possible can bring some advantages, by reducing resistance and decrease the distractions (like having to "imagine").
Complaints are to be expected. "It's just not working for me!" so did you attend our training? "The system goes crazy with this function!" but did you follow our instructions?
Testers often act like, or, deliberately place someone untrained in the UAT for some reason (they may claim these are the unbiased testers). Yet apart from the real errors and bugs, time could be gone into explaining basics and troubleshooting false alarms.
And especially for this reason, these should also be captured in the log with appropriate classifications. Later even if someone brought up the similar unconscious negative comments, perhaps based on the impression from others, yet without any precise descriptions... there is something recorded and help defending.
Passing the UAT stands as a project milestone. But often that is a client managerial decision where their concerns or perceptions overrule the assessment result.
Questionnaires collected from the testers should help as a supporting evidence to push the management forward. Yet the majority of responses are often found either blanked or picking the medium rating. Since people are slow heaters whereas serious feed-backs are hereby found precious, it is truly nonsense not to design the questions in the way people are guided to answer.
In fact any survey, or feedback questions should be working like a marketing medium, through which people are reminded those benefits and values of the system could have brought them, through which people are comforted because there is always a work-around and those defects are really not the show stoppers, and eventually our target (conclusion) message of getting the UAT passed is also conveyed.