Many young testers who have just passed their ISTQB certification exam are rather surprised when they find out what real life has in store for them. Instead of detailed and meaningful specification documents from which they can straightforwardly derive their tests, they are often confronted with poor documentation or even monstrous legacy systems for which no documentation at all exists.
Currently some of these systems, which have been running since the 70s at Austrian government institutions, are being ported to more modern platforms. The testers are supposed to verify that the ported versions of the systems are working exactly as the old ones. But how can they do that if the knowledge about the thousands of functions and business rules is hidden inside the heads of a handful of specialist users? What’s more, how shall the developers of the new systems know what to implement?
Faced with these challenges, we came up with a method to learn about the SUT and create a test model from which tests can be generated at the same time.
- Let the specialist users perform typical use cases on the old system (in our case an IBM 3270 emulation) and capture everything they’re doing and saying with video software. We used VLC together with the UScreenCapture extension as recording tool because it’s simple to use and offers a good balance between video quality and file size.
- Based on the recordings, create a model of the use cases with a model-based test (MBT) tool. We used Atos TEMPPO Designer for this purpose. It allows the user to create Activity Diagrams and to include business rules. Step sequences that appear more than once can be defined as reusable and parametrizable building blocks. For instance, if the ‘Login’ sequence appears in a lot of test cases, it should be defined as a building block with the parameters ‘User’ and ‘Password’. At the start, the modelling is tedious work, but the more reusable blocks you have, the faster you get.
- From the model, let the MBT tool generate test cases in the format of your choice (Excel, XML, HP ALM…). The generator will consider the business rules and try to find valid paths through the model until it has achieved the desired coverage. You will end up with a set of test descriptions that cover all use cases and variants that you specified. These descriptions and of course the model itself are also a valuable input for the developers who can use them as a design specification. Thus, our approach can be seen as a form of test-driven development.
- Once the ported version of the SUT is available, capture its user interface with a GUI test automation tool such as HP UFT. The resulting object map can be imported into TEMPPO Designer and the GUI objects assigned to the steps in your model. Based on this information, the tool is able to generate complete, executable scripts for HP UFT. Simple actions such as clicking a button are translated directly into the scripting language. If complex scripting is required, constructs such as loops can be entered and maintained directly in TEMPPO.
- If some requirements are changing in the SUT’s next version, the effort for test maintenance is minimal. Instead of adapting every single test case, you just need to change a few building blocks in the test model and let the tool re-generate the tests.
While our approach requires some effort particularly at the beginning of the project, it has several advantages that in our experience soon compensate the initial investment:
- All use cases are documented on video with minimal effort for the specialist users
- The detailed test specification is a valuable input for testers and developers alike
- Clear separation between high-level use cases and technical details
- Simple creation of variants of a test case, e.g., for different user groups or platforms
- Systematic test coverage by the generation algorithm
- Automated generation of executable test scripts
- Significant reduction of test maintenance costs because of reusable building blocks
We have been using MBT in a large number of projects, e.g., at the European Space Agency. In the course of a case study we found out that the initial investment in MBT paid off after about 4 test cycles and brought huge savings afterwards. For details, see chapter 9 in Experiences of Test Automation by Dorothy Graham and Mark Fewster.
