Test Process improvements are driven by the context. In our search for the optimal test approach, we assess the systems, the development methodology or the risks that we cover. However, there is a much bigger factor that we often overlook. This factor has a great influence on how we improve and what we improve.
Imagine you’ve got a new colleague. Not a junior, but an experienced professional who has worked for several companies. She started two months ago, enthusiastic and full of expectations. She is open minded like one often is in a new job. She is hopeful that some things are organized better than within her previous job. One day, the manager calls her with him. He asks her what things she noticed about the way the testing is done in the department. Each organization has its problems and every building is has cracks, so no doubt she has seen some suboptimal testing. He likes to use her objective look and challenges her to suggest some improvements. What do you think she will tell? What did she notice? Would that be the same things that annoy you? What measures would she suggest to improve them? Would she come up with the same solutions as you would?
Let us repeat this thought experiment in a slightly different setting. This time the new colleague is not a woman, but a man. He has got just as much of experience, but has a very different track record. After two months you organize a session, just like you manager did in the previous experiment. You ask you colleague the very same questions. How does this session go? What did he notice? Would you react the same on his proposals for improvement as your manager did?
During the Belgium Testing Days 2014, nearly a year ago, Huib Schoots, Jean Paul Varwijk, Fiona Charles and I staged an experiment very much like this. During an interactive session we asked the participants to think in groups on improvement proposals that certain types of testers would come up with. It became clear that the person and previous experience are decisive for the things that stand out to him or her and get noticed.
Role play session: Jan Jaap Cannegieter and Isabel Evans discuss possible test improvements during our session on the Belgium Testing Days 2014
Why does one care about the dried plants on the windowsill, while the other is focused on the security policy of the company? Why is one disturbed by the ad hoc mentality in the organization, while the other is happy that not a day in the office is the same? Personal preference determines what we notice, what bother us and what improvements we propose. This personal preference is influenced by our believes and experiences. Some improvement we believe to be good even though we have no empirical evidence. Sometimes we suggest an improvement, because we have seen it work elsewhere.
In the second part of our experiment, we simulated the conversation between the supervisor and the new employee on the basis of a role play. A beautiful setting, especially because we chose two actors with an opposing worldview. The talks quickly became discussions where the participants tried to outdo each other with substantive arguments. Yet, one has to remember that it often not the content that determines the outcome of the discussion. It is the receiving party, which has its own frame of reference. That’s why you see that a security risk and your opponent is only worried about that parched plants. His observations differ and thus his reality is different. His experience and believes are different and he will therefore have a very different picture of what improvements are most urgent.
The experiments lead to valuable insights. The context determines the improvements we need to make. The systems, the development methodology and risks drive or choices only to a certain extent. They are overshadowed by our personal observations and preferences. This explains why we sometimes feel like we are running against a wall and feel misunderstood. It explains why we (if we are the manager himself) are disappointed with the proposals put forward our employees. By recognizing our own coloring and preferences, we are better able to understand our colleagues. We can connect with their experiences and judge their suggestions for what they are worth. The context still determines the improvements we will make, but we better understand the context. The context is… us!
