I have recently been testing a new device (containing embedded software) produced within our company as a replacement/development of a previous device. Now this device has been in the offing for a long time now, and I was fortunate enough to be given a copy of the requirements document a long time ago. This spurred me on to start thinking about requirements in software testing.
Requirements in Software Testing
Don’t get me wrong – requirements in software testing are a great thing. In fact, I would go as far to say that without some form of specification of behaviour testing is impossible. At a recent workshop, we were asked to “test” some black boxes. I said this was impossible without being told the requirements, as the requirement might be for an object that did nothing that I could throw at someone. Without requirements of some sort, it is impossible to check whether the actual behaviour observed is required, a lucky accident, or a bug.
The point I am making here is that despite developing an extensive test suite from the requirements I was given, they have been virtually useless in testing the device. The reason? As the software is being delivered incrementally, only parts of each test that I developed are currently applicable at present and these parts are intermingled with un-implemented functionality. Perhaps with hindsight I should have pursued with the developers if this would be the case, but at the time it was presented as a “new” product with “new” requirements.
Models of Software Testing
In reality, what has been more useful at this stage is to use the “old” device being replaced as a model of what the new device should perform. In my experience so far, I have not found models of the software test process helpful (I’m not saying that they are not useful). However, I have found modelling the system under test to be very helpful. In fact, requirements can be regarded as one model of the system. Changing the model that we are using is a good way to reveal tests that have been missed.Been using solely requirements? Try a state transition diagram. Or some other way of describing all or part of whatever you’re trying to test.
That’s what a model is – a way of describing (thinking about) all or part of a system. Seen like that, it is easier to think of requirements in software testing as just one way of modelling, and that there may be others that may show test cases that the requirements model has missed. Happy modelling! Happy testing!