Taxonomy of Software Testing Terms

Home Forums Software Testing Discussions Taxonomy of Software Testing Terms

Viewing 17 posts - 31 through 47 (of 47 total)
  • Author
  • #3588

    @Keith – I guess it’s a relatively small sum of money but my point was that it appears people are not prepared to spend it in order to access the standards. I doubt the sum is the sole barrier to testers and organisations using these standards?


    @Stephen, Stuart Reid was involved in both the original British standards and the ISO 29119 standards. I’m going to get in touch with him to ask for his views on this conversation.


    Stuart actually presented a webinar on ISO 29119 last year: View it here


    A taxonomy works best the items in it could be considered as coordinates for nuggets of useful material, so that any single item could have few coordinate values in place, and then same item could be listed on all relevant places where the material is relevant. Hardest would be the artificial constraint where the taxonomy is assumed to be a single hierarchy and that each material would be relevant on only one point of hierarchy / locatable via only one dimension only. For example there could be checklists for social aspects of test planning: they have both testing life-cycle releveance, relevance to “Test management” and also in “People” dimension. As there really (and fortunately) is no single testing approach guiding we all to assume certain hiearchy in our heads, so there also is no single dimension to have all relevant classifications in it as one value, but more like items float in multi-dimensional space and several coordinates are potential ways to find the material.

    It looks like the Non-funtional testing is a subset of Acceptance testing (a simple copy paste error?). I’d suggest to adjust the upper organisation as follows:

    Testing through the SDLC (avoiding acronymes would be better here, I suppose, like “Testing through the software life-cycle”)
    (Each waterfall phase / test level listed here, from Requirements to Maintenance)

    Testing types
    Functional testing
    Non-functional testing

    Then from testing methods onward it seems mostly logical, except 2 things:

    1. Only place I find questionable is the “results analysis” alone with no context. This points out the missing of the lack of plan-driven testing continuum, or testing life-cycle. Testing life-cycle in all its artificial simplicity is great for organising different testing practices (methods / tools / techniques). For example the TMap had these 5 that encompass useful concepts and logical continuum of important testing activities and decisions — as long nobody assumes they follow each other linearly or that all should be put into some document 🙂

    Testing life-cycle
    Planning & Control

    2. The division of testing techniques and methods is IMHO somewhat artificial and they might be grouped as “Testing practices”.

    BTW, if the proposed changes in all this thread posts have been implemented somewhere else than the original post, how about giving the link on this thread?

    Cheers, Erkki


    Some more classifications of testing methods or kinds of testing (might overlap with threads shown above)

    acceptance testing
    accessibility testing
    aggressive testing
    ad hoc testing
    agile testing
    all-pairs testing
    alpha testing
    API-based testing
    arc testing
    automated testing
    back-to-back testing
    basis path testing
    behavioral testing
    best representative testing
    beta testing
    big-bang testing
    black-box testing
    bottom-up testing
    boundary value testing
    branch condition combination testing
    branch testing
    build verification testing
    business process-based testing
    claims-based testing
    code-based testing
    combination testing
    compatibility testing
    complete testing
    complex testing
    compliance testing
    component integration testing
    component testing
    concurrency testing
    condition determination testing
    condition testing
    configuration testing
    confirmation testing
    conformance testing
    context driven testing
    control flow based testing
    conversion testing
    data driven testing
    data flow testing
    data integrity testing
    database integrity testing
    decision condition testing
    decision table testing
    decision testing
    design-based testing
    development testing
    deversified testing
    dirty testing
    documentation testing
    domain testing
    dynamic testing
    efficiency testing
    endurance testing
    equivalence class testing
    evaluation based testing
    exhaustive testing
    expert testing
    exploratory testing
    extreme value testing
    feature or function integration testing
    field testing
    finite state testing
    follow up testing
    functional testing
    functionality testing
    generic testing
    glass box testing
    gray box testing
    guerilla testing
    incremental testing
    installability testing
    installation testing
    integration testing
    integration testing in the large
    integration testing in the small
    interface testing
    interoperability testing
    invalid testing
    isolation testing
    keyword driven testing
    LCSAJ testing
    link testing
    load testing
    logic-coverage testing
    logic-driven testing
    logic testing
    long sequence testing
    maintenance testing
    maintainability testing
    manual testing
    maturity testing
    migration testing
    modified condition decision testing
    modified multiple condition testing
    module testing
    multiple condition testing
    mutation testing
    N-switch testing
    negative testing
    non-functional testing
    operational profile testing
    operational testing
    pair(ed) testing
    partition testing
    path testing
    performance testing
    portability testing
    program testing
    prototype testing
    random testing
    recoverability testing
    recovery testing
    regression testing
    regulation testing
    reliability testing
    requirements-based testing
    resource utilization testing
    risk-based testing
    robustness testing
    safety testing
    scalability testing
    scenario testing
    scripted testing
    security testing
    serviceability testing
    session testing
    site acceptance testing
    smoke testing
    specification-based testing
    standards testing
    state transition testing
    statement testing
    static testing
    statistical testing
    storage testing
    stress testing
    structural testing
    subject-matter expert testing
    subpath testing
    sympathetic testing
    syntax testing
    system integration testing
    system testing
    thread testing
    top-down testing
    unit integration testing
    unit testing
    usability testing
    use case testing
    user acceptance testing
    user scenario testing
    user testing
    volume testing
    white-box testing


    Hi Erkki, thanks for your input! Can I summarise your suggestions as follows?

    1. Testing through the sofware life-cycle
    to include:
    Testing (as a lifecycle phase)
    Results Analysis
    Acceptance Testing

    2. Testing Types
    to include:
    Functional testing
    Non-Functional Testing

    3. Test Practices

    to include Methods and Techniques as above

    4. Test Process as above

    5. Test Automation as above

    6. Testing Artefacts as above

    7. People as above

    Seven overarching areas like this would be a great starting point for getting a taxonomy that works on this platform but if we can apply relevant tags to all content resources, then the idea of material being discoverable by different coordinates becomes possible.


    Thanks Rainer, 159 terms I count there. I wonder where they would sit in a taxonomy as opposed to the alphabetical list? I also wonder how many more terms are commonly used? Have we come to a near exhaustive list?

    I’d also be interested to get the thoughts of the few hundred people following this thread but so far not joining the conversation too 🙂


    @Paul, Erikki’s puts forth a schema that is reasonably harmonious with ISO concepts for product (i.e., systems and software) development and reasonably harmonious with the concept of a taxonomic hierarchy where one employes generalization and specialization. Your thread has surfaced the fact that a taxonomic schema isn’t universally understood for its role in a vocabulary of a domain discourse.

    Even in the light of the recent turmoil and dissension regarding ISO/IEC 29119 by key principles in the Test community, the concepts of classification developed over the years by ISO/IEC should not be ignored. IMHO

    ISO 15288 “Life Cycle” “stages” of a system = Concept, Development, Production, Utilization, Support, Retirement

    During the “Development” stage the following technical transformation processes may be executed = Stakeholder Requirements Definition, Requirements Analysis, Architectural Design, Implementation, Integration, Verification, Transition, Validation

    During the “Development” stage the following project governance processes may be executed = Planning, Assessment and Control, Decision Management, Risk Management, Configuration Management, Information Management, Measurement

    During the “Development” stage the following project-enabling processes may be executed = infrastructure management, human resource management, quality management.

    I tend to place “processes” as a central concept in my thinking. Why? Firstly, because I am a Systems Engineer. Secondly, because it is through the execution of a process that transformation occurs. It is how value is created. Processes define “What”, they do not define “How”. “How” is defined in the lower elements of the behavioral hierarchy by process activty, tasks, methods, and procedures. Next, I have structure (e.g., specifications, architecture descriptions, designs, reports, roles, etc…) as these things are inputs and outputs of processes (i.e., technical transformation, project governance and project-enabling). This is how I might start at building a taxonomy. But this is what I am comfortable with, it is how I think. It is how I have been conditioned to think of my domain, mostly through my exposure to ISO/IEC standards.

    This figure is an attempt to illustrate how I think about a process and its elements Process Concept Model. This view of the model is not complete, it is merely one perspective of my process concept model.

    Hopefully one perceives the harmony with classification ideas put forth by Erikki. They are not equivalent, but they are not discordant either. “Test Artifacts” are structural elements. “People” relate to Roles. “1. Testing through the software life-cycle” relates to processes in general. Methods relate to @Erikki’s concepts of testing practices and behavioral aspects of testing. The “How” of a process.

    , you state in your reply to @Erikki “discoverable by different coordinates” which suggests, to me, that an entity might be discoverable by way of more than following a hierarchical classification tree. Was this your intent? If it was, then aren’t you really speaking of an ontology now? A much more complex concept, at least to my understanding.


    Geoff, an ontology is a stretch too far I think! What I meant was that a piece of material would be discoverable in more than one way – it would be found at relevant points in the taxonomy and not just one.


    Given the interest here in the topic of a standard language, it would be good to get your thoughts on the ISO 29119 standards – Ronan has started a discussion on this. Why are some testers so vehemently opposed to this while others are ‘ok’ with the development of standards?



    I have been working in the past year or so on a taxonomy in regards to software testing information but I believe it can be applied in general for software testing.

    It is is a visual shape: The Testing Map

    If you consider it useful I can also provide you the whole tree structure. There are around 300 areas on the map.

    Adam Hepner

    I have a quiet proposition: this form of discussion makes me feel a bit… dizzy – it’s hard to follow, does not let itself be linked properly, and change propositions are not really properly managed. The developers have perfect tool for managing collaborative discussion and changes to text documents. It’s obviously GitHub :). I’ve created a quick repository here: (it is under creative commons, and I will initiate it with content from this discussion in the coming days). I encourage everyone to clone the repo, provide additional explanations, and submit merge requests and/or issues. Together we might make it one of the best repositories of the knowledge in the web!


    This is a great idea Adam and you are right, the conversation has many tangents. I look forward to the next version.

    Ronan Healy

    I thoughts this was an interesting article by Donald Firesmith on taxonomy of software testing. He has divided it up into a number of different parts.

    Jeba Qpt

    It involves different phases such as Initial/Planning phase, Requirement analysis phase, Design phase, Coding phase, Testing phase, Delivery and Maintenance phase.
    Initial Phase involves gathering requirements by interacting with the Customer.
    Requirement analysis phase involves doing detailed study of the customer requirements and judging possibilities and scope of the requirements.
    SRS (System Requirement Specification) will be created in this phase
    Design phase involves dividing the whole project into modules and sub-modules by doing High level Designing (H.L.D) and Low level Designing (L.L.D).
    Coding Phase involves creating source code or program by the programmers by referring the design document.
    Testing Phase involves getting clarification for the unclear requirements and then writing test cases by Testing Team based on the requirements.
    Delivery & Maintenance phase involves installing the application in the customer place and providing the details such as release notes to the customer.
    Read and learn about


    When I first read Donald’s paper, his taxonomy didn’t sit well with me. Flattening the multiple dimensions the way the taxonomy does just doesn’t seem to be the right way to conceptualize test. Each to his own.

    The contributions of Jeba and several others are excellent examples of the challenges of this proposal. ISO standards have already established system, software and test vocabularies, yet it is clear from several terms used by others in this thread that these international standards are not having the influence they should be having. I suspect it is mostly due to the “pay to play” business model that ISO, IEC, and IEEE observe. The expense of accessing these standards is prohibitive for most. While I do not assert ISO standards to be the “Holy Grail”, they have a place in the ecosystem.

    We are all entitled to our opinion, but whether that opinion is acknowledged as having relevance and value by our peers is another matter.

    Ronan, Has there been any real movement on your proposal?


    Hi all,

    This discussion petered out somewhat I guess – it’s pretty daunting but I think @stevean’s suggestion is a good starting point:


    A Glossary is a good starting point. Thrash out all of the terms and short definitions.
    This can then be expanded in two routes. 1 – A Wiki to take the definition and explanation further and 2- a Model to show when and how to use each term.
    I think in Testing a Taxonomy is limited as it’s a bit 2 dimensional. A full model could be multi-dimensional showing all the contextual relationships and enabling the user to identify the appropriate terms and methods for each project and situation.

    If there’s genuine interest in this being a member-driven project a glossary of terms with short definitions sounds like a good place to start. Any volunteers to get us started?

    We could start by having terms defined in this thread and then branching the conversation into different threads as it grows?

    Open to other suggestions of course.

Viewing 17 posts - 31 through 47 (of 47 total)
  • You must be logged in to reply to this topic.

One Response to “Taxonomy of Software Testing Terms”