Exploratory Testing in Practice.

[Martin]

A couple of very interesting looking tutorials announced at this year’s EuroSTAR programme for 2015

It’s a shame they clash!

I would very much like to hear about people’s practical experiences of testing in an exploratory fashion.

Has anyone come across any good tools or adopted tools that help them test in exploratory fashion.?

How do people go about managing exploratory test processes, planning/adding/refining/automating/reviewing?

Are they any good practical resources for exploratory testers out there?

Thanks
Martin

[Stephen]

To be honest I’ve never been in a company that either allows proper exploratory testing, or has enough people with the skills to perfrom exploratory testing correctly in a disciplined way.

I’ve tended to use standard test tools, e.g. Test Management tools, and as I perform the test write it up directly into the tool, recording the pass and fail.
Steps would be. Identify a list of test titles that help to guide coverage and direct the tester (Myself) to the areas I wish to exercise. Load these into the tool.
As I explore each title I write the test up as I go. Sometimes duplicating the test with different paths, experiances, etc. Sometimes removing a test title as it gets covered by another test.
Screenshots get added to the ‘script’ and sometimes I would use ‘CamStudio’ to record video: Because sometimes you can only tell the story with video.

What amazes me is that this is exactly what my collegues writing formal scripts do, but they do it up front and then re-run the tests during a manual test run.

I may have missed the point of exploratory testing, and if so, please let me know as I’m always keen to learn and improve.

[Darshit]

Hi Martin

Yes, we do practice exploratory testing within Agile development team. First let me know do you have Visual studio 2013, if yes it has excellent tool for
exploratory testing. otherwise you can use manual Test charter document to manage your session for testing for the user story (PBI). After sprint planning meeting we do following steps for exploratory testing.
1. We write test charters (scope of testing for the user story) while developers are designing user story.
2. We review test charters with developers before they start coding on the user story (so that they get an idea of our scope of testing)
3. Then we review and break Test charters in multiple so that one test charter bounds to 60 min.
4. Once we get user story delivered for testing, we use test charter to record actions, test notes and defects if we found any.
5. Also for regression, we use original test charter as reference while recording notes on new Test charter sheet.

I will send a sample of test charter sheet, have a look, normally it should target to one functionality to be tested. Also I will send few guides which will help you to understand exploratory testing briefly
hope this will help you.

Thanks

[Nuno]

I think the biggest tool you can use is your imagination. The beauty in exploratory testing is in gaining knowledge of your application and reacting when something unexpected happens.
Having experience in the typical problems that happen makes your live easier in experimenting ways that your application may break but I don’t believe that you can come up with all the ideias beforehand. The ideias should flow when you are doing the testing itself

[Darshit]

Exploratory Testing

Exploratory testing is simultaneous learning, test design, and test execution

In other words, exploratory testing is any testing to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.

Exploratory Testing is a Profoundly Situational Practice

• Have you ever solved a jigsaw puzzle? If so, you have practiced exploratory testing. You pick up a piece and scan the jumble of unconnected pieces for one that goes with it. Each glance at a new piece is a test case. Still, can you imagine what it would be like to design and document all your jigsaw “test cases” before you began to assemble the puzzle, or before you knew anything about the kind of picture formed by the puzzle?

When we solve a jigsaw puzzle, we change how we work as we learn about the puzzle and see the picture form. If we notice a big blotch of colour, we might decide to collect all the pieces of that approximate colour into one pile. If we notice some pieces with a particularly distinctive shape, we might collect those together. If we work on one kind of testing for a while, we might switch to another kind just to keep my mind fresh. If we find we have got a big enough block of pieces assembled, we might move it into the frame of the puzzle to find where it connects with everything else. Sometimes we feel like we are too disorganized, and when that happens, we can step back, analyze the situation, and adopt a more specific plan of attack. It would be silly for us to carefully document these thought processes in advance.

• Instead of asking what test we are instructed to run, exploratory testers ask what’s the best test I can perform, right now?

• These factors changes continuously throughout the testing of project. The power of exploratory tests can be optimized throughout the test process, whereas scripts, because they don’t change, tend to become less powerful over time. Whereas scripted tests, once you’ve executed a scripted test one time and not found a problem, the chance that you will find a problem on the second execution of the script is, in most circumstances, substantially lower than if you ran a new test instead

Practicing Exploratory Testing

• An exploratory test session often begins with a charter, which states the test mission and perhaps some of the tactics to be used.
• The only official result that comes from a session of ET is a set of notes about the product, failures found, and a concise record of how the product was tested.
• A basic strategy of ET is to have a general plan of attack, but allow yourself to deviate from it for short periods of time. Similar to “sightseeing tour bus”, the key is not to miss the tour entirely, nor to fall asleep on the bus.

Facts:
• Exploratory software testing is a powerful approach, yet widely misunderstood
• All testers practice some form of exploratory testing, unless they simply don’t create tests at all. Yet few of us study this approach, and it doesn’t get much respect in our field. This attitude is beginning to change as companies seek ever more agile and cost effective methods of developing software.
• Microsoft practices a formalized process of exploratory testing for the purposes of certifying third-party applications for Windows compatibility http://www.satisfice.com/tools/procedure.pdf) and session-based test management http://www.satisfice.com/sbtm) is a method specifically designed to make exploratory testing auditable and measurable on a wider scale.
• Exploratory testing is not against the idea of scripting, In Some contexts, user achieves better testing through scripted approach; in other contexts, benefit more from create and improve tests as you execute them (explore). Most situations benefits from mix of scripted and exploratory approaches.
• Remember, in ET we make maximum use of skill, rather than attempting to represent every action in written form.
• If you think about it, most formal written test procedures were probably created through a process of some sort of exploratory testing.
• Exploratory testing can be described as a martial art of the mind. Well, you don’t become a black belt by reading books. You have to work at it.

Happy practicing.

[Arifa]

Hi Martin,

I use mind maps to keep a track of my exploratory testing sessions. I would recommend to have time bound sessions for exploratory testing.

[Jesper]

Hi Martin,

A source regarding exploratory testing practices can be found here: http://www.ministryoftesting.com/resources/exploratory-testing/.

With regards to tools – anything could do. Session-based test management was conceived while using Test Director / quality Center. Also it’s a matter of the formality of your documentation. screen recorders could be one way to do it.

Consider to have a backlog of test ideas, select some, complete then to a certian criteria (time?) and select more.

Maybe test design “on the fly” is what fit’s your content, or maybe it’s the heristics of the Rapid Software testing course

Jesper

[Rikard]

Exploratory testing is involved in any testing you do.
And in my experience, many of the most important problems are found outside of what I explicitly was looking for.
So an open mind that understands areas to explore further is what is needed.
Documentation-wise I use the same system as for other tests, I just test more than what was written in advance.
I document what others need, and keep my own notes in a text editor.
For the interested, I have made a video of a 20 minute exploratory session that is available here:

You probably need to invent your own style, for your unique situation.

[Christina]

Good reading material/resources have already been mentioned – so here are my 2 cents from my experience:

How you manage exploratory testing and which tools you use must depend on the organisation you work in and the processes and boundaries that you work within. What kind of information from the testers is needed for a go/no-go decision? How heavy on documentation, how metric-happy and process-adhearence-strict is the company? How much are you able to influence or change the way the organisation works?

For many reasons my organisation is heavy on traceability, documentation and metrics, and use a heavy-weight tool for test management (Quality Center) – yet we have managed to tweak our testing processes to allow for a more exploratory, less scripted approach within some of the teams. This is how we do this:
– We use mind map, Word or other tool to document our testing ideas.
– The Mind map is uploaded in QC for documentation.
– We create test cases in QC based on testing ideas (mainly headlines supplemented by some notes – normally no steps).
– We track the testing of the testing ideas by “passing” or “failing” each test case (i.e. test idea) as the testing progresses.
– Bugs/issues are recorded in QC defect module and logged against the test case (i.e. test idea).
As TM I can then report testing progress with some metrics that are at least to some extent comparable between our onsite (partly exploratory) testing and our offshore (scripted) testing in a way that the management can understand. BUT always supplemented with a lot of information about issues, what works, and what doesn’t – which I get from discussing with the testers very often.

So my suggestion to you would be to do some (more) reading, and to analyse your organisation to see what makes sense in your context, with the organisational restrictions and possibilities that gives you, and then work from there.

/Christina

[Martin]

Hi Everyone!

I just wanted to thank you all for you most interesting and helpful replies to this forum. I know it has been a while since I last replied but I am still very much in the process of taking on board all of your replies and looking into how I can adopt exploratory testing into my organizations testing strategy as we seek to work more agile.

Darshit, I do not have Visual Studio 2013 available but I am making enquiries through our IT department to see if I can get hold of a license. I will let you know if I have any luck!

Thanks everyone!
Martin

[Alex]

Exploring the software to discover its functionalities and drawbacks is what does. So, this is a formal testing process that doesn’t rely on test cases or test planning documents to test the application. Instead, testers go through the application and learn about its functionalities. They then, use exploratory test charters to direct, record and keep track of the exploratory test session’s observations. [formated mod/JO]

[Swashtika]

Thankyou for sharing this useful information for more detail you can undergo this blog also

Exploratory Testing, A Guide Towards Better Test Coverage

[Author]

Posts