The following article is a follow-up to a submission I put forward for EuroSTAR 2014. It explores part of my journey so far in software testing, and why I feel we need to look at the way communicate with our customers….
Now that I have your attention, what do I really mean? Well, I am looking for an answer to this question:
“Is the Test Strategy the best way of communicating what we do to our customers?”
By ‘Test Strategy’, I am also including Test Methodologies and other large multi-page process documents.
So, why ask the question?
Getting to this point has been a journey for me over the last few years. Like a river, it has meandered along, changing direction every once in a while. And like a river, it has absorbed tributaries into the overall flow as I see, hear and experience different things during my testing career.
So far, those tributaries are:
1. If we see Testing as a business, what type of business would it be? Who would our customers be?
2. Who reads the Test Strategy once written? How much effort is expended keeping it up to date to match the changing nature of the project?
3. How do you assess the quality of a testers work?
What type of business is testing?
If testing was seen as a business, it would come under the service provider category as we don’t make anything. In this case our customers would be everyone that has an interest in the testing we are doing. This would include developers, stakeholders, project managers etc. However, we are also customers in what is a symbiotic relationship e.g. without developers building the application we would have nothing to test
For more in-depth analysis of this, please read the following article:
http://www.ministryoftesting.com/2014/05/understanding-relationship-testers-customers/
Is the Test Strategy relevant to our customers?
How often do you write a complex test strategy document at the start of the project and struggle to get it ‘signed-off’? How much effort have you put in to keep it up to date as the project has progressed? Or did you decide that the effort wasn’t worthwhile?
If this is the main method of saying what we are going to do on a project, and we’re finding that our customers don’t find it relevant to them shouldn’t we be looking at this and asking: “Why?”
How do you assess the quality of a testers work?
I have deliberately used ‘assess’ and not ‘measure’. I don’t believe the quality of a tester can be measured. i.e. using metrics.
What metrics would you use? The number of tests run per day? The number of bugs raised? In both cases, it would be like trying to compare apples and pears: some tests could be quicker to run than others and one part of the application could be ‘buggier’ than the other. I don’t think it would be a fair comparison.
If we follow the testing as a business model, then the quality of our work is really about the service we provide. In order to assess that, we need to agree and set a benchmark on the level of service we will provide to our customers.
This leads to another point: what about the value testers bring to the project?
I have been involved in, and read, discussions on what value testers add to the product or application. My view: testers don’t add value to the product. They can help in either protecting the value the product will provide to the business or trying to ensure the product will meet its potential. I think testers can add value to the project as a whole when they use their skills in assisting other members of the project team. E.g. testers with technical knowledge can assist developers by doing code reviews or even with writing the training manual or running training sessions for the users.
Navigating the Thames or Liffey
This has been the hardest part for me: trying to bring all the ideas above into one coherent whole has been like trying to navigate through the currents and eddies on a river.
My first step was to go back to looking at testing as a business and look at how other service providers set a benchmark with their customers. A lot of transport companies have passenger charters, doctors have the Hippocratic oath, and solicitors have a code of conduct. When I started exploring this idea further I initially thought of a charter. I then heard about session based testing and the use of ‘test charters’ within that form of testing, so that was discarded. I didn’t feel a ‘code of conduct’ would be strong enough, which is when I thought of a Test Contract. To me, a ‘code of conduct’ would not cover the two-way nature of the relationship but a contract would.
Instead of focusing on what we will do on a particular project, the Test Contract would focus on the service the testers will provide for their customers. Just as importantly, it also needs to cover what testers need and should expect from their suppliers to give the promised level of service. In effect, it would be project neutral as it would be documenting the relationship between testers and their customers. This should make it more meaningful for our customers as what is being documented is more tangible to them.
Managing the customer-supplier relationship
To do this, I see the contract containing a set of Promises, Expectations and Needs for each customer-supplier relationship. E.g. between the testers and development team or the project manager etc.
A Promise is the commitment from the testers to provide a service at an agreed level.
e.g.
We promise to raise bugs with the agreed level of detail for the developers to be able to reproduce the issue quickly.
We promise to report the test status in the agreed format.
The difference between an Expectation and Need is one of emphasis. They describe what testers should expect or would need from their suppliers to meet the commitments as promised.
e.g.
We expect bugs raised to be treated professionally.
We need bugs to be fixed in the agreed time-frames.
We expect to be kept informed of changes to the schedule or scope of the project.
The contract can then be used to assess the quality of service the testers are providing by using each promise as a benchmark to be compared against i.e. did we fulfil the promises we made and are their mitigating factors where we didn’t?
What do you stand for as a team or as an individual?
Going back to the charter or ‘code of conduct’ concept, other service providers give a statement on what their core values are as a business. I think if testers did something similar, it would make it easier for our customers to understand what we are trying to achieve on and for the project.
If the contract is to be on behalf of the whole testing team, then all team members should have an input to it: everyone needs to buy in to give it any meaning.
Keeping it relevant
The Test Contract is a living document. It is documenting the relationship between people. Over time, these relationships will change either due to personnel changes or naturally due to time. So the Test Contract needs to be reviewed and updated to ensure it is still relevant both to the testers and their customers.
