Michael Bolton was the joint winner of Best Tutorial at EuroSTAR 2018 and we were lucky to record a webinar with him for Higher Value Checking. Interested in his answers to some of your burning questions? Have a read below.
Have you ever had to deal with a situation for a management who wanted to automate all testing and get rid of manual testing completely? Did you manage to persuade them otherwise? If so, how?
I had a wonderful experience with that and a CTO I was having dinner with. I said, what do you want to do with all the testing? He said, I want to automate all the testing. And I said, you can’t automate all the testing any more than you can automate. All the management testing is an intellectual process. Testing is a process. It requires human minds. It requires exploration. Investigation. What the manager is saying in that case is I want to use tools in powerful ways. I think that’s what the really want to do. I would recommend as a way of preparing yourself for this is to look at a paper that James Bach and I wrote in 2016 called a context driven approach to automation in testing. Uh, and there’s been a lot of talk about automation in testing since we uh, wrote that paper in beforehand.
We have for years advocated talking about automation in testing and not about test automation. Another thing that we emphasize is we don’t talk about manual testing and automated testing because testing can’t be automated. Remember that slide testing is learning about a product, evaluating a product by learning about it through exploration and experimentation. And machines don’t evaluate machines, can’t ascribe value to something because machines don’t understand intention. Machines don’t learn. Machine learning, as I said, is a metaphor. Machines do not explore. They don’t experiment. What is the arguably the greatest accomplishment of humans in terms of exploring unfamiliar territory of exploring spaces? It’s, it’s the Mars Rovers. The Mars Rovers Explore Mars. Some people say, no way. The Mars Rovers don’t explore Mars. They don’t even know they’re on Mars. You never hear the or explorers saying, that’s one small step for a robot, one giant leap for robot. We explore and we use these phenomenally powerful, sophisticated, elegant tools to extend and enhance our capacity to explore.
But it is us. We humans who explore just like we humans manage, manage projects and nobody ever talks about automating the product management. Nobody ever talks about that. Nobody ever talks about automating the CTO role. That’s not going to happen. That’s because tools don’t understand intentions. And similarly, if we could get a tool to test our products, we can get tools to write them too. And that doesn’t happen. Nobody talks about automating all the programming. So don’t talk about automated testing. Don’t talk about manual testing. There is testing and there’s testing using tools. And the fact that testers used tools is unremarkable. Nobody talks about manual research. Nobody talks about manual journalism. Journalists use tools, researchers use tools, we use tools, but we don’t automate those things. If you want more help on that, that’s a great thing to have a chat with me about. I, I help people learn. That’s what we do in rapid software testing. We help people learn about how to test, talk about testing, and think about testing in powerful ways and really does start with the way we talk about it.
Where can I get a hold of your publication with James Bach,
https://www.developsense.com/courses/RapidSoftwareTesting.pdf
Have you ever encountered a situation where developers were reluctant to test at higher levels than unit level? Have you managed to fix that issue? And if so, how?
Well more time. Uh, developers don’t like to add tests that are the top level by and large. Now this is not, this is not a pathology, I don’t think because developers are in a different mindset from testers. Developers want to make your problems go away and develop it. To do that requires a mindset that’s ambitiously focused on, envisioning success on synthesis, on making things, on building things. It’s the builder mindset. It’s the maker mindset. Testers by contrast ought to be, it seems to me focused on finding problems. So as a developer, you’re natural impulse and you, it’s really hard to, to get out of this, your natural impulse is to focus on how the product is working, how it’s solving the problem. It’s cognitively difficult. It takes time and it takes effort to, uh, switch from, I’m putting this thing together to help you solve your problem too.
All right, how might I not have done that? That’s just a really hard thing to do. It’s not impossible, but it goes against the rather optimistic, intentionally optimistic green of programs to do that. And that’s why I think it’s a really important thing to have at least one and more people who are permanently in the testing mindset who are not focused on envisioning success but who are instead focused on anticipating failure, who are focused on finding problems, who are focused on being critics rather than makers. And I think the collaboration between those two mindsets and those two skillsets are what makes for a powerful team. Diversity is not just about, different genders and different cultures and different ethnicities and different educational backgrounds.
Those things are important. But I think what those things help us get different temperaments and different mindsets. And the testing mindset is an important one to have on the product. Instead of seeing the fact that developers are disinclined to a test in the ways that, that, that testers might be inclined to test, treat that as a feature and use the developer’s skills to aid testing rather than to do testing. Let we think of testing like a villa where other people are welcome to come in and we’ll do the clean-up work. We’ll do the dirty stuff we’ll wash the dishes and we’ll mop the floors and we’ll take care of the lawn and we’ll patch holes in the house and so on and so forth. Other people can be our guests in that filler and um, uh, we can enjoy it a great party together. The theme of the party is finding problems, but you know, we’ll do the dirty work of that and we’re happy to do that.
Do you have any experience with quick check, property-based testing? Is that testing or checking to you, allowing a computer to generate scenarios beyond what we can imagine and just checking a specific properties
I do to the degree that the process incorporates a configuration, operation, observation, data generation, followed by the application of decision rules followed by the reporting of the outcome of those decision rules, alerting humans to that. To the degree that that’s all done entirely, automatically, entirely algorithmically. That’s checking. But what I think somebody is asking about there is something that’s still nonetheless must be embedded in testing. So, checking is included in testing and it sounds like the user has an interesting tool on his or her hands in order with which a lot of variation can be induced by which a certain kind of problem can be detected by which the system can be a disturbed & shaken up in various ways. That’s to the degree that that’s being done entirely by the machinery might be checking or it might be using a tools in order to generate that data.
But you know, it’s okay, that’s all happening inside a testing process because once again, the human is actually doing the evaluation of whether that check is reporting a legitimate problem or a problem. It doesn’t matter. Humans are doing interpretation and humans are exhibiting the intention to do that and initiating the use of that tool. So it almost doesn’t matter. What I would say is simply that we’re using tools in powerful ways with something like that. And that’s cool. That’s a great idea. It’s humans that remain in control. It’s humans who are doing the learning. It’s humans who are doing the evaluation, the exploration. We’re using tools to aid us in doing that. That’s great.
We are delighted to have Michael back with for this year’s conference with his tutorial ‘Using Risk to Guide Testing‘
Would you like to learn in person with Michael and 60+ speakers at this year’s conference? Then have a look at this year’s programme.
Sign up to be part of our community and never miss out on great content again!
