Practices & Attitudes in Software Testing

[Paul]

Hi all, last year we conducted a 15-question survey on practices and attitudes in software testing – this is the first phase of an extensive look at how testers perform their roles and their attitudes towards their work.

We’ve just published the report and you can download it here.

I’d like to get input from testers – what are your thoughts on the report?

And, as we prepare for phase two – what questions do you want answered?

Looking forward to your feedback.

[Tim Western]

There are a number of ‘suspect’ issues with the questions. For example a person may be qualified for a job by experience or other, non certification based training (Question 9 and also impacts on how the user understands Question 11.) It might be more useful to ask what things a tester believes qualifies them as a tester and list certifications as just one option.

Question 1 and 2 really bug me because, most Agile models do not use the Watefall Process model (of which a LOT of articles lately seem to suggest that the SDLC is waterfall, when that isn’t always the case. There are multiple software development models of which waterfall is one. Some others include: Build – Fix Model, The Spiral Model, the V-Model, SCRUM, Kanban, Extreme Programming, Test Driven Development, Rapid Prototyping, Iterative/Incremental Development (which was an early form of Agile methodologY I believe). I’m sure I’ve missed some.

By trying to pinpoint when testing happens by giving these phase names you are implying something in the survey about how most software is made. A lot of software has and is still made with the Waterfall mode, but there are a lot of other models in use as well. So I’d prefer questions that help reveal at what stage of a process (agnostic of a specific process model) that a tester is fully involved as opposed to cherry picking these from Waterfall:

Requirements
Design
Development
testing
Deployment
Maintenance

The second question, btw also goes off the rails for the same reason.

The question about types of testing a person does is interesting, but I feel a question about whether a user authors, writes, or maintains test automation scripts and perhaps what level they are at (Framework Development, BDD/Cucumber/Gherkin, Integration, Unit, etc.) would be useful in helping to dig into things. I suspect you’d find that people who write test harnesses may be more likely to write Unit Tests, then more high level tests.

Question 4 is a question I’d love to see repeated in Phase II. Very good question to ask. (I loved the analysis of this one question later on which shows, as you expect, the older, more experienced, or longer with a company testers are more likely to be concerned about this area).

Question 6 confuses me, because some of the answers imply that Developers and Managers are being asked questions about testing. (47% are dedicated testers while 88 are Full-time according to this? I wonder what the purpose of digging in with the type of hire a person is serves though.) I’m not saying this is a bad idea to ask this, I wonder if there’s confusion in what is meant by Full Time.

The combination of the job titles gives me pause. It might be interesting to know what % are titled as tester vs QA, and it might be interesting to know whether testers are happy with their current job title, or would prefer a certain job title be used when referring to their work.

I mentioned my concerns with 9 and 11 before, my concern is that people who have learned testing without taking a certification are getting lumped in with those who have, and may be painting a false picture of the testing landscape for those who want to presume qualification == certification.

I do like the efforts went to in the analysis, unfortunately because of the questions I’ve raised, most of the data I feel is useless in understanding the state of the industry.

I hope this helps.

Some things I’d definitely like to see ask.

What a tester feels is there primary responsibility

collaborating with devs on acceptance criteria
authoring and executing hands on test cases
development of test automation scripts
maintenance of test plans
maintenance of test automation.
etc.

I’d ask them whether they feel the industry is improving in the areas of testing or not also. A survey like this is a good point to get the feelings about the state of the art.

[Paul]

Hi Tim, thanks for your detailed and constructive feedack. Some of your suggestions mirror some of the feedback we’ve received offline (at EuroSTAR 2014) – there are limitations to responses where assumptions are made e.g. that Waterfall is the default model for software development.

What do you think of the idea of segmenting phase two and looking at different areas of testing e.g. test automation as separate segments?

Tim Western wrote:

The combination of the job titles gives me pause. It might be interesting to know what % are titled as tester vs QA, and it might be interesting to know whether testers are happy with their current job title, or would prefer a certain job title be used when referring to their work.

Can I ask why this interests you?

This preliminary report has it’s limitations, that’s clear from feedback but I guess we were ‘testing the waters’ to see what level of interest there is in a more considered, detailed study of testing practices. The interest is clearly there, we just need the testing community to steer it with us.

I look forward to hearing more.

[Tim Western]

Well there are lots of job titles that testers may fall under. I’ll list a few.

QA Engineer
QA Analyst
Software Tester
Software Test Engineer
Software Developer – Test (could also count for Software Development Engineer in Test)
Business Analyst
Test Automation Engineer

I do think, job titles matter to an extent, and they have an impact on how people perceive you. Many of us think the entire idea of calling Software testers QA is wrong, misleads people into thinking they have a department responsible for something, that really they have little real control over. There are actually two aspects to this that could be important, one is to figure out how the industry is titling testing professionals, the other is to figure out how many years experience people are at with certain titles. I know some who feel there isn’t much room to grow or move in testing, and I think the Job Title is a key indicator of where the industry is.

[Author]

Posts