The blog post summarizes the work done during the start-up of a large IT project for a Norwegian Governmental organization. The aim is to visualize the combined effects from different QA and SQA (test) activities to the management in order to show the assumed overall positive effects on project quality.
During the start-up we had to produce a project steering document clarifying QU and the proposed testing process. The project was assumed to be executed Agile within an organization that had only some experiences with Agile and structured test.
Based on the work of Capers Jones I was able to present in objective numbers were and how the different SQA activities would attribute to the overall project quality which we defined in terms of defects found during the first 6 months of operation.
How to sell in the necessary development processes and a structured test regime?
This was the question I faced during a period I was responsible for Quality Assurance and Test during the initiation phase of a larger project. As I was working for a (Norwegian) governmental organization I became involved in a project were we had to deliver a steering document to the financial department in order to get the necessary funding for the project. This steering document should, besides an estimate of the costs, contain the plans and procedures how to manage and control the project through its phases.
The hosting organization, my employer, had hardly any formal IT development procedures or routines available and was struggling to implement a testing regime (my job). Without any formal assessment performed, I considered the IT organization on a CMMi level 1. The aim for the project was to develop the system using agile methods but again also here, there was hardly any proven knowledge or competence.
In the team who was preparing the document I became responsible for test and quality and had to deliver the plans and convince my fellow team members that these where the way to go for the project. Besides this I became involved in the estimation team trying to estimate the costs for developing the code/system and proper release plan.
It is off course the wet dream for every test manager to become involved so early I a project and being able to come up with a first draft of a testing strategy. Very generic, through enough, but setting a baseline for further development.
During the thinking and writing of this testing strategy I did write some short white papers to the team about testing, quality, requirement management and configuration management and I was able to write down how these processes were interacting and depending of each other to deliver the maximum quality to the system.
At that time I was already known to some of the work of Capers Jones and decided to buy his book on Software Cost Estimation in order to help the team with some objective figures. As I was paging the book (only over 600 pages) I came across some statements and figures about the correlation between the number of defects a project would introduce in a system depending on its maturity and the number of defects the project would be able to detect and remove based on the same maturity.
As a former Software Quality Engineer in a CMMi level 3/5 organization, I began to relate the project’s maturity with the necessary development processes and the quality assurance measures needed. This enabled me afterwards to design and develop a “system” of test strategy, development plans and quality assurance measures which would enable the project to develop the system without to many defects and discover and remove the most of them.
So how does it fit altogether then? QA is about preventing that defects occur or are implemented in the system. Test is about enabling the project to remove most of the implemented defects.
Preventing is done with:
- describing (development, QA) processes with the necessary control mechanisms
- defining templates and quality checklists for the most important plans and artifacts within these processes
- performing all the defined quality assurance activities.
I decided to describe the following processes (IEEE 12207/15528):
- Development process (agile)
- Configuration management (including change management)
- Requirement Engineering and Management
- Verification and validation (test management)
- Quality assurance
Defect removing is done with:
- defining the test process in a test strategy
- specifying the test activities (with a focus on static testing and validation)
- perform all these test activities
With this (documented) system I was able to present the percentage of defects I was able to remove from the system before going life and to define how many critical defects we would deploy.
I was also able to calculate the costs of all these activities and visualize the effects of not doing QA or not performing all test activities.
