About 6 months ago, while I was working with team of test managers and we were discussing the challenges they faced and what they could do to mitigate or avoid them. This inspired me to think about common mistakes software test managers make and the challenges they face. In the discussion we broke these challenges down in the following truisms:
- The Business ALWAYS wants to go live yesterday
- Development will never DO ENOUGH testing for your liking
- Due it its nature, test execution is always the last thing in the project life cycle so the focus is always on YOU
- The environment is ALWAYS BROKEN
- Everyone wants to BLAME TESTING FOR EVERYTHING
What was apparent to me was the fact that these challenges haven’t really changed over the past 18 years I’ve been involved in testing.
This prompted to consider why, when so much of the testing we do now has evolved, the challenges and associated mistakes test managers make still seem to be the same, and to ask the question why we keep making these mistakes?
Some Unscientific Research
Before going in this in more detail, I thought it would be good to see if what I was experiencing was valid, so I asked a number of experienced IT managers and Test Managers the simple question – What are the biggest mistakes a test manager can make?
While the results are by no means a perfect reflection, they do bear out the same thoughts I was having.
Testing Art and Testing Science
Out of my research I broke down the challenges into 2 groups:
1. The art of test management – focused on the soft skill elements of being a test manager, which is often the bit no one really teaches you about – How to communicate and deal with the other groups of stakeholders
2. The science of test management – focused on the test management process, or “what do the numbers tell you” this is often easier to teach and there is lots of training courses (like ISTQB) that can teach you this part.
To be a good test manager you obviously need to be good at both and they are both interrelated with each other. For example, understanding what your metrics mean in terms of risk, but not being able to communicate it, isn’t going to help the project.
Based on this I came up with the following common mistakes:
The Art
1. Accepting what others think – I think often in the heat of a project we get swept along with the need to go live or move to the next stage, when we really know the quality doesn’t justify that progression.
2. Not leading… But following – related to the accepting what others are thinking, we often don’t speak up and show the leadership when it’s required. When this happens we are often marginalized or ignored, because it’s believed (wrongly) we don’t have anything new or important to say.
3. Not communicating “the impact” – One of the biggest issues we have even when we do see an issue, is making people understand what the impact of the course of action is going to be on the quality of the application.
4. Losing sight of the big picture – When we are in the middle of testing and things are happening and 100 miles an hour, its often easy to lose sight of the big picture and how our defects and the applications quality will impact that. Remember that at the end of the day we are developing (and testing) software to solve a business problem, and that’s view we need to come back to.
5. The ability to negotiate – Often these issues are not black or white so there will need to be some give and take on both sides of any issue, and we need to be able to negotiate will other parties to get what we need.
The Science
1. Not being able to use metrics to prove your point – Where possible we need to be able to use metrics to demonstrate the issue we are facing. As testers we have lots of metrics available to us from straight defect metrics and test case execution metrics, to more complex ones like defect removal efficiency and cost of quality.
2. Not implementing the basics – Don’t forget the basics of good test strategy these include Early engagement, Test early on requirements and using risk as your guide.
3. Not accepting that your strategy needs to change – at the end of the day projects change and morph. I don’t think I have every worked on a project where the requirements were locked down 100% the day I started testing. So your strategy and approach to testing needs to evolve too.
My Top 10 Commandments for Test Managers
1. You are not always going to be friends with the development – but you need foster mutual respect
2. The business / program managers aren’t always right – but they do need to see you as their trusted advisor on quality
3. You’re the expert & leader when it comes to testing so fight your corner if you are right
4. Your main job is to be there to communicate risk
5. Collaboration on the big decisions – Make sure everyone understands the issue and the impact & get sign off on your strategy
6. Don’t be afraid to re strategise your testing…. use leading indicator metrics to help drive your decisions
7. If things are going wrong call it early and often… until someone listens
8. Lies, damn lies and testing metrics – there is more to metrics than the open defect count
9. Make sure you get the basics right – Test as early as you can & implement a risk based approach
10. Things never go to plan… a good dose of pragmatism and flexibility is vital
So hopefully this gives you a good guide to common mistakes software test managers make. If you can think of any more, please add them in the comments below.
