A new format for test cases

[Jesper]

Usually a test case consists of a description with preconditions, and a number of steps that each must be PASS|FAIL (ie “the script”) and comes with “Expected result” and “actual result” up front.

I’ve been in contexts where these steps where to be taken very literally requiring photo proof and in context, where they were “more like guidelines” (to quote a pirate). But in both contexts all I could think of was that each step should have a disclaimer, that “nothing else matters”.

Pharma contexts would request the test case to FAIL completely if the expected result and actual did not match perfectly, even if something “better happened” etc. I don’t believe that you can plan and predict everything.

I would prefer to articulate test cases as questions. Previously I would probably articulate them as actions Nouns & Verbs. “Order a DSL line, with these and these add-ons for this and this customer profile”. But we can consider so much more by writing: “Can a DSL line with this&that be ordered to this customer profile” (see Asking Open Questions )

The steps, with only a linear approach, doesn’t handle as many end stated as often is the case. So I have been considering that test cases could perhaps follow the format of exceptions  from programming languages like Java.

TRY – some things you do try, that might fail

CATCH ERROR – a specific error and and action to resolve it

CATCH WILDCARD – catch anything other unknown error and resolve it

FINALLY – something to do if try section is error free.

How about I wrote my DSL test case steps like:

TRY { look up the customer; setup the order; add options }

CATCH { options not available} THEN { FAIL test case }

CATCH { customer cannot be looked up } THEN {just find another }

CATCH { * } THEN { call for 3 amigo’s to discuss}

FINALLY { DONE }

I know it’s quite a challenge to reconsider reformatting testcases. But I see that there is a need to make more outcomes of the business transactions explicit and make room for every step to have more than just a single outcome.

 

 

 

 

 

[Archana]

This is a pretty interesting approach to writing test cases.

Applying this is all situations may not be possible. But it may prove helpful specially in those cases when the test cases are to be followed word by word.

[Kasper]

Although I sympathize with the idea I can’t really find a good use for it in testing software  systems.
In the end computer instructions are only 0 or 1, false or true. Introducing ambivalence in testing just does not seem like a good idea. I can see big troubles ahead in large systems I have worked on: fire control systems, command and control systems, medical systems, and even big data warehouses with client data.

[Jesper]

The whole idea of writing testcases in another form is a challenge. Many companies and testers have a vetted interest in status quou.

So yes this format may not be easy to work with.. especially in contexts where testing is more about compliance and validation that the requirements are exactly implemented. But as mentioned, elaborating what has been considered would provide better transparency and express the implicit observations found during testing. .. if anyone is interested in this..

IT solutions in my opinion are less and less about binary transactions, but supporting business activities that people do. on one hand.. and on the other hand more automation, more AI, more algorithms. If they are binary truth/false based, why do we discuss the <span style=”color: #000000; font-family: NeueHaas, sans-serif; font-size: 21px;”>algorithmic morality?</span>

But that’s a side track…

As we often do while testing. Side track.. consider other things. Evaluate observations both implicitly and explicitly. Im looking for a form that removes the implict “nothing else matters” / implict “everything considered equal” – and am looking for a way to articulate this as part of the creation of the tests.

 

[Author]

Posts