User Story Acceptance Tests have become fairly standard in Agile development but are easily confused with much-different User Acceptance Testing (UAT), which then tends to be omitted, thereby likely reducing test effectiveness (see also Unconventional Wisdom V15: Test Effectiveness, https://huddle.eurostarsoftwaretesting.com/unconventional-wisdom-v15-test-effectiveness/).
User Story Acceptance Tests confirm the implemented code satisfies the user story and thereby presumably provides desired value. Test-first unit tests confirm the code works in the manner the programmer intends. Typically both are automated and together constitute all the dynamic testing executed in a sprint. Somewhat less frequently employed, pair programming is a form of static testing that further aids the quality of software developed in Agile. These few testing techniques have come to dominate Agile testing, while at the same time generally without much awareness pushing out other types of testing, such as User Acceptance Testing (UAT).
Conventional Wisdom
Many Agile projects and practitioners believe test-first unit tests and User Story Acceptance Tests, possibly also along with pair programming, together suffice to assure the quality of code developed in an Agile project.
To be fair, Agile’s explicit embedding of these tests within development often results in more and more-timely testing than occurs in many traditional projects, where testing may be left until the end when it’s often shortcut or simply skipped due to lack of time. Moreover, it’s well recognized that fixing a defect takes longer the later it’s detected. Thus, Agile does find certain defects earlier, when they are easier and cheaper to fix.
One Little Word
I sense that many folks believe User Story Acceptance Tests are the same as User Acceptance Tests (UAT). The common tendency to omit “story” and refer to User Story Acceptance Tests as “User Acceptance Tests” or just “Acceptance Tests” exacerbates confusion of the similar terms; and it causes true UAT to drop from consciousness.
In fact, that little word “story” makes User Story Acceptance Tests far different from and much smaller than true UAT. A user story describes a small piece of functionality, usually in the form of:
As a <type of user, role>
I want, need <something>
So that <I get some benefit or value>
User Story Acceptance Tests are intended to demonstrate the code satisfies the user story. That’s a bigger scope than a test-first unit test, but still very small.
In contrast, UAT is an end-to-end test of much larger functionality. Because Agile focuses on sprints, which in turn focus on user stories, it tends to lose sight of testing larger pieces, especially integration and end-to-end tests.
Moreover, UAT is intended to demonstrate that the finished system will work when used in a realistic manner by real users in a real production environment. As such, UAT effectively exercises code that could be considered to be addressing numerous user stories which together carry out business processes. UAT focuses on the full business process rather than any of the individual user stories contained within it.
Note that User Story Acceptance Tests not only are not the same as UAT but also cannot accomplish UAT’s purposes. Besides focusing on small pieces of functionality in isolation, they are automated and thus cannot demonstrate realistic usage by real users. Moreover, the programmer generally does the automating, which shifts attention from the user’s view to the programmer’s.
Conventional Agile wisdom assumes demonstrations of finished code serve as UAT. Anyone who’s ever seen a demo, typically by a salesperson, should appreciate that it’s hardly a test. Demos are entirely from the developer’s perspective and intentionally avoid showing anything that may be a bit shaky—exactly opposite true UAT’s purposes.
UAT’s Issues Make It Easier to Omit
In my experience, true UAT generally has few defenders against being skipped. UAT is widely considered an unpleasant high-effort low-payback activity. Typically, it dumps a lot of last-minute effort on users to do testing they’re not experienced or interested in, on top of their already-high normal workload. Then UAT still misses many defects, for which developers have been known to blame users quite vociferously.
While such experiences are common, I find they stem from misunderstanding UAT and are not necessary. It shouldn’t be surprising that users usually have no idea what to do or how to do it for UAT. Unfortunately, developers seldom help the situation. Perhaps you too are familiar with directives to users such as, “Here’s my code. Go play with it to try it out and don’t do anything stupid.” People play with toys. Code is not a toy!
Professional testers tend to lead users astray too, largely because they apply mistaken models and try to shape users in their own image, which few users aspire to. UAT is often somewhat of an afterthought, ordinarily initiated at project end, after independent testing is nearly completed and the project probably is already late.
Most testing is what I call “technical or developmental,” intended to demonstrate that the code conforms to the design. That’s important and necessary testing, but it’s often compromised when testing merely reacts to what’s been coded. Typical last-minute “try it out” UAT invariably is inefficiently unplanned, unsystematic, and ineffectively merely reacting to the code instead of the design.
More importantly, it’s not what UAT should be doing. Most software errors are in the design. Appropriate UAT should be demonstrating that the design and code are what the business needs. Repeating the professional testers’ thinking and techniques just performs another technical or developmental test that code conforms to design, not that the design is right. No wonder users don’t warm up to it.
I think you’ll find my Proactive User Acceptance Testing™ is a better approach that makes users confident, competent, and cooperative. By applying special techniques to plan and design Proactive UAT™ early for execution at the end, testing drives development to produce better software from the start and also has better more thorough tests that catch more of the remaining fewer defects.
Look for the next Unconventional Wisdom blog post on Proactive UAT™. See how it fits with prior posts on Proactive Testing™ Common Misconceptions https://huddle.eurostarsoftwaretesting.com/unconventional-wisdom-v8-proactive-testing-part-1-common-misconceptions/ and how it Drives Development https://huddle.eurostarsoftwaretesting.com/unconventional-wisdom-v9-proactive-testing-part-2-drives-development/. Contact me for direct advisory assistance and training.
“It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so.”
– Will Rogers
Welcome to my Unconventional Wisdom blog. Much of conventional wisdom is valid and usefully time-saving. However, too much instead is mistaken and misleadingly blindly accepted as truth, what I call “conventional we’s dumb”. Each month, I’ll share some alternative possibly unconventional ideas and perspectives I hope you’ll find wise and helpful.
