Exploratory testing is way past its introductory phase. I remember when applying it for the first time in 2007. It was a perfect pilot. In that specific project and old legacy system was replaced. Due to the legacy there was no documentation. I remember the team having little to no time to prepare for the tests. Fortunate the employees all had a lot of knowledge of the replaced system. We decided to solve the testing problem by using exploratory testing. After a shaky start, we found a good modus operandi. During the evaluation all team members were enthusiastic about the new approach.
I remember doing this project during a training of James Lindsey (See for example a EuroSTAR video from James). I signed up for this training because I wanted to benchmark my experience. I was especially curious about how exploratory testing had evolved and wanted to know if there is some sort of industry standard. My conclusion: it is not really there. Exploratory testing can be done in several ways. Everyone seems to tune the concept on the basis of his or her own experiences and context. Anything goes!
Although exploratory testing is with us for some time, I still meet a lot of testers that are quite new to the concept. When I had to introduce exploratory testing to such a group at a customer site recently, I did not want to send them into the real world with a mere “Anything goes”. In my course I therefore introduced four ways (modus operandi) on how to apply exploratory testing.
Exploratory testing as a mindset
First, exploratory testing is a mindset. Testers are not constrained by scripts. They will always look for weak spots in the system. Testing is an intellectual challenge in which we examine the system and observe its behavior critically. We try hard to outsmart the system and to find combinations of actions and inputs that trigger unpredictable behavior.
Deviate from the trail
Second, leave the beaten path. There are still organizations in which whole armies of testers execute their predefined test scripts to the letter. This scripted checking is the opposite of exploration. In this later we test outside the script. If we think it will lead to better tests, we follow our gut feeling and invent our own tests. In traditional organizations this can be introduced as a Friday afternoon ritual: ‘Guys, put your test scripts aside. We’ll see if we can break the system. A bottle of wine for the best bug.’
Play before your script
Designing your test upfront has the disadvantage that the test cases can be based upon assumption more than on real system characteristics. An efficient use of exploratory testing may therefore be to get to know the system before you design your tests. This can be called “first play, than script”. Perform an exploratory run on an early version of the system. This has some advantages: we pull testing forward in time. As the system is still being developed, our preliminary findings can effectively be processed. Also, the testers get a feel for the system and they can define more targeted tests.
Session-based pair testing
Fourth way of doing ET is Session-based pair testing (SBET). This variant is invented by James Bachand is the most complete modus operandi. It is therefore my favorite. With SBET the testers work in pairs and execute test missions. They are defined on so called test charters. These are defined in advance, but during testing new ones can be added to the pile. The charters are assigned to the testers based on their priority, so the most valuable are run first. After a test session the testers meet for a debriefing in which they share their findings and challenge each other with critical questions. Charters that are competed with confidence, we can tick off as done. This makes the test progress visible to management.
The four modus operandi can be mixed and are not that far apart, as they may seem. The notes taken during the tests session are a nice prelude to the regression test scripts. Session-based pair testing can be part of the Friday afternoon ritual. A right testers mindset is always a must, no matter what testing you do. So the four ways are not independent of each other. But that is OK. As long as they help to better understand exploratory testing and lower the threshold for applying it. I am sure if you try it, you will be just as excited as I was – and still am- in 2007.
BIOGRAPHY
Derk-Jan de Grood was chosen as this year’s European Testing Excellence Award winner at the 2014 EuroSTAR Conference. He works for Valori as senior test manager and product manager. His drive is to improve the visibility of testing in both agile and traditional organizations, by sharing his knowledge and experience by means of training, presentations, workshops and publications. He is a regular speaker at conferences like EuroSTAR, wrote several successful books on software testing and publishes articles for the major testing magazines.
