Taxonomy of Software Testing Terms

Home Forums Software Testing Discussions Taxonomy of Software Testing Terms

Viewing 30 posts - 1 through 30 (of 47 total)
  • Author
  • #3412

    Hi everyone, we’re looking to create a taxonomy of terms which we will use to make resources on this website more easily discoverable by relevant subject matter. This taxonomy could also be used as a point of reference for areas of members would like to see us focus on for new webinars, eBooks, presentations etc.

    The goal is to create a bank of content that can be efficiently searched by testers trying to solve a particular problem – whether directly related to testing or indirectly e.g. advice from others on managing people; communicating to stakeholders etc.

    I’m not a tester, so I doubt this list is fully complete / logical / relevant (and maybe not even useful!) but who better to help improve it than a community of professional testers?

    List to follow shortly, I look forward to your replies – any input welcome!


    Testing through the SDLC

          • Requirements Analysis

            • Requirements/Specification
          • Design

            • Test Planning/Design
          • Development(Coding)

          • Testing

            • Functional Testing (Correctness Testing)
              • Unit Testing
                • Smoke Testing
                • Sanity Testing
                • Regression Testing
              • Integration Testing
                • Component testing
                • Back-to-back testing
              • System Testing
                • Model-Based Testing
          • Results Analysis

            • Reliability Achievement and evaluation by testing
          • Deployment

          • Acceptance Testing

            • Non-functional Testing
              • Acceptance Testing (Functional)
                • UAT
                • Operational Acceptance Testing
                • Interface Testing
                • Contract & Regulation Acceptance Testing
                • Installation Testing
                • Alpha & Beta/Field Testing
              • Availability Testing
              • Baseline Testing
              • Compatibility Testing
              • Compliance Testing
              • Documentation Testing
              • Localization & Internationalization testing
              • Performance Testing
                • Load Testing
                • Stress Testing
                • Reliability
                • Stability Testing
                  • Endurance
                  • Soak
                • Spike
                • Isolation
                • Volume Testing
                • Configuration
              • Recovery Testing
              • Reliability Testing
              • Resilience Testing
              • Scalability Testing
              • Security Testing
                • Penetration Testing
              • Static Testing
            • Maintenance

          Test Methods

                • White Box Testing – a.k.a. logic-based testing; glass-box testing; design-based testing; structural
                • Black-Box Testing – a.k.a. data-driven; input/output driven; requirements-based testing; functional testing
                • Grey-box testing – a mix of white and black box methods

                Test Techniques

                      • Tester Intuition/Experience
                        • Exploratory
                        • Ad hoc Testing
                      • Specification-based
                        • Equivalence partitioning (black-box)
                        • Boundary-value analysis (black-box)
                        • Decision table (black-box)
                        • Finite-state machine-based (black-box)
                        • Testing from formal specifications (black-box)
                        • Random testing (black-box)
                      • Code-based
                        • Reference models (white-box)
                        • Control flow-based criteria (white-box)
                        • Data flow-based criteria (white-box)
                      • Fault-based
                        • Error guessing (black-box)
                        • Mutation testing(white-box)
                      • Usage-based
                        • Operational profile
                        • Software Reliability Engineered Test (SRET)
                      • Nature of Application
                        • Object-oriented testing
                        • Component-based testing
                        • Web-based testing
                        • GUI Testing
                        • Testing of concurrent programs
                        • Protocol conformance testing
                        • Resting of distributed systems
                        • Testing of real-time systems

                      Test Process

                            • Waterfall
                            • Agile
                            • Test-Driven Development
                            • Behaviour-Driven Development
                            • Top-Down Testing
                            • Bottom-Up Testing

                            Test Management

                                  • Attitudes/People
                                  • Test Strategy
                                  • Test Process
                                  • Test Documentation
                                  • Test Team
                                  • Cost/Effort Estimation
                                  • Test Reuse & Test Patterns
                                  • Test Planning
                                  • Test Case Generation
                                  • Test Environment Development
                                  • Problem Reporting/ Test Log
                                  • Defect Tracking

                                  Test Automation

                                        • Code-driven Testing
                                        • Graphical User Interface Testing (GUI) Testing
                                        • Software Testing Tools
                                        • Metrics

                                        Testing Artefacts

                                              • Test Plan
                                              • Test Design
                                              • Traceability Matrix
                                              • Test Case
                                              • Test Procedure
                                              • Test Script
                                              • Test Schedule
                                              • Test Suite
                                              • Test Data
                                              • Test Harness


                                                  • Behaviour
                                                  • Coaching / Training
                                                  • Certification
                                                  • Collaboration
                                                  • Communication
                                                  • Creativity
                                                  • Critical Thinking
                                                  • Diversity
                                                  • Heuristics
                                                  • Innovation
                                                  • Test Experiences
                                                  • Leadership
                                                  • Teamwork/Team Building
                                                  • Recruitment
                                                  • Skills

                                                  This is scary. It just goes to show that over time the industry has over thought everything and keeps adding to the list rather than rationalising it.

                                                  i think the idea of the list is excellent for several reasons. Including the original intent of providing a reference sieve for definitions.

                                                  But also to highlight the plethora of terms and methods and concepts that are in use so that we can rationalise.

                                                  I bet iso21199 doesn’t even cover this list 😉

                                                  How are you going to manage updates and suggestions for the taxonomy? Wiki or some other resource


                                                  Hi Stephen, I think the forum would be a good starting point for new suggestions. Just list them right here, if there is enough interest to create a wiki then we will do that for sure, great idea!

                                                  The list itself seems unwielding, so rationalising it makes sense. Where to from here?


                                                  Hi Paul, It’s a good effort bring the things in structured way. Better make repository place ( MediaWiki ) with the help of admin, so every one contribute their knowledge on it in the more structured way.


                                                  Nice initiative Paul! You can think about adding few more terms mentioned here:
                                                  – “Build” along with Deployment as we frequently use it.
                                                  – Test environments – QA, Development, Staging, Inactive, Production etc.
                                                  – Scrum as another test/ development process.
                                                  – Test Estimation and Test Management tool in Test Management.
                                                  – Automation tool in automation.

                                                  I will get back to you if I remember anything else.


                                                  Also, you can add Geo-beta along with alpha and beta.


                                                  Test Data: masking, anonymous,
                                                  Artefacts: Test Policy
                                                  Test management: Risk analysis


                                                  People: Motivation, Performance Management, Delegation of tasks, Self Initiation
                                                  Allocation of Resources: Time, Cost, Quality, monitor progress and correct
                                                  Consultancy Skills
                                                  Business Relationships Analyse user requirements, Advise users on scope and recommended test approach, Discuss options for operational improvement, Develop client relationships
                                                  Test Management : Change Control
                                                  Industry Awareness


                                                  I’m surprised no-one has mentioned the V-Model or W-Model yet. 🙂

                                                  @Afreen what’s Geo-beta? That’s a new one on me.

                                                  One of the problems with a taxonomy is the application. It’s not a 2 dimensional list. i.e. Some items are related, but not relevant when used together with unrelated items.

                                                  e.g. talking about Shift left in an Agile/Scrum world is not appropriate as you can’t shift very far left in a 2 week sprint.

                                                  And we need to start defining the terms also to people know and understand what they mean. Starts to look like an ISTQB certification, but hopefully not an ISO Standard 🙂

                                                  To add to the list:
                                                  Verification – Tests and Checks against the requirements and specifications aimed and what we have done what we said we’d do.
                                                  Validation – Tests and Check against the customer expectations, i.e. does the product satisfy the customer/end users needs. Not the same as acceptance testing, although acceptance testing is part of validation.
                                                  VVRM (Verification and Validation Requirements Matrix) – Part of Requirements tracing. A matrix of tests/checks mapped to requirements; illustrating coverage of requirements by tests during design and completeness of requirements during execution.
                                                  Test – An exercise intended to investigate the target (Unit, Interface, Feature, Functionality, System, System of Systems, Process etc) to provide information on suitability to release to the next stage. This is an intelligent investigation guided by scripts but not limited by them.
                                                  Check – An inspection of the target (Unit, Interface, Feature, Functionality, System, System of Systems, Process etc) to assert a specific condition, Either positive or negative. A check has a defined input and output.
                                                  System under Test – The system that is the target of the test, as apposed to any auxiliary or connected tools/systems used during the test.

                                                  It is amazing how many items you can add to a list when you get started. It’s a bit like a crossword. The more you stare at it the more answers you fill in. If I spent all day at this I still would think of more terms I’ve heard or used.

                                                  It’s funny really. I think of Testing as an easy job with lots of nuances and something new every day. But when you compile a list like this it’s not simple at all, it’s very complex. No wonder we have difficulty communicating when we have all of this running around in our head, just waiting to be expressed and used.


                                                  Stephen, Geo-beta is not commonly used by all. It is a stage where product is made live on some countries or a group having just enough users to collect data, but not the target market. Beta is released for a larger group of users from where market size and revenue related decisions can be made, but again not the actual target market. Beta and geo-beta can have the same meaning for many of the companies though.


                                                  So the list continues to grow! I might have opened a can of worms here!!

                                                  @Padmaraj – the wiki sounds like the way to go. We will factor that into the next round of changes to this site which will commence fairly soon.

                                                  @Vivi, @Afreen – than you both, keep them coming!

                                                  What I would like to do is create an exhaustive (or near-exhaustive) list and then agree on the most common sub-groups by which material on this site would be categorised. The list itself could be updated/maintained as a wiki to include definitions as @Stephen mentioned.


                                                  So @Stephen, where would your suggestions sit in the current list?

                                                  VVRM – in Requirements?
                                                  Verification? Functional Testing?
                                                  Validation? Is this a synonym for Non-Functional Testing or something else?
                                                  V-Model & W-Model – Test processes?

                                                  Tests – can apply to several steps along the SDLC? This is less likely to be used as a tag for website content but another term to be defined.
                                                  Checks – similarly to tests, they are not confined to one particular step in the SLDC? This is less likely to be used as a tag for website content too but another commonly used term.
                                                  System under test – this is the system being tested? Again, less likely to use as a tag but something that should be contained in a wiki of terms and definitions.

                                                  From the outside looking in, testing definitely appears complex and I think the terminology plays a part in that. I don’t think it would be easy to get to grips with the coining of new phrases and acronyms that continue to appear. However, if there was a means of understanding the relevance and context of these terms then there’s a greater likelihood of testers being able to share their experiences and seek out relevant knowledge that does exist (somewhere) rather than assuming their problem hasn’t been solved before.


                                                  Incidentally, @jane-moller-nash mentioned on email that the ISTQB may have a similar list to this – has anyone got a link to it? For some reason I’m having trouble accessing their website today!



                                                  VVRM is a Test Management tool. Should be Built during the setup phase, updated and managed through the Design Phase, Monitored and used for reporting during execution and used in the final reports during exit/closure phases.

                                                  V and W Model are Test Process.

                                                  Verification is during Unit, Integration, System testing. Functional Testing.
                                                  Validation: Non-Functional for me means Stress and Performance Testing. But as an antonym for the Earlier phases known as functional I suppose Validation is a Synonym for Non-Functional. This is were terms get confusing and need clarification.
                                                  Validation should be done at all phases of the project. i.e.
                                                  Validate that the requirements meet the customers needs, and not just the contract.
                                                  Validate that the integration meets the customer needs, particularly if your integrating to 3rd party or customer systems, and that they don’t just meet the requirements and specs.
                                                  Validate that the system supports the business needs not just meets the requirements and specs.

                                                  Validation is about how the customer will use the product, not about how it was designed and built.

                                                  @Afreen: Thanks for the explanation of Geo-beta. It seems to me to be a name for a way of phasing a Beta roll out.


                                                  Some more terms:

                                                  Common terms used in testing: Attack(negative testing), bug, defect, Defect density, COTS, entry/exit criteria, incident management
                                                  Testing: Confirmation Testing, dynamic testing
                                                  Test Management: Configuration Management, Defect Management, Defect life cycle, Version Control
                                                  Tools & Techniques: Debugging (tool), Simulation, Decision Coverage


                                                  Thanks @Jayanthi, that ISTQB list appears to be an alphabetical list of known terms. How they relate to each other is where it gets interesting.

                                                  @Stephen, thanks for the explanations!

                                                  I wonder is it common for testers to use a large subset of this or are there smaller subsets that would be common place in different organisations/regions.

                                                  For example, would everyone working in testing in a large multinational use the same terminology regardless of where their testing team members might be located?


                                                  @Paul lol, “use the same terminology regardless of where their testing team members might be located?”

                                                  I work in a multinational, multi-cultural, multi-phased company. And however much I try to promote and enforce a common taxonomy, people will interpret their work differently. I also find some terms are contextual, e.g. depending who you are and what your role in the who SDLC.

                                                  e.g. Acceptance Testing and UAT.
                                                  For an Engineer/Developer acceptance testing is done by the testing team and internal users. So their deffinition of UAT and who does it refers to internal users and the team/group recieving their output. They don’t always have a concept of view of the end user. If they do, it’s CAT, Customer Acceptance Testing. Or Operational Testing.

                                                  For the System Test Team UAT is conducted purely in the test environment and is conducted by testers who try to simulate users.

                                                  Then there is the operational team and various delivery teams who are customer facing and refer to UAT as testing with the customer and end users.

                                                  At least 3 different definitions of UAT, depending on your postion and outward view. I am constantly trying to educate all teams on a common use of the term and the correct terms from a holistic perspective. So that everyone knows what is meant when each term is applied.

                                                  One problem is the sales cycle not engaging with The test practice early enough and inventing their own terms. Once the term is set with the customer you have to maintain it for constency and appearance that your company does know what it’s doing. Again I’m fighting to get a common language established.

                                                  So any attempt to provide a central, industry standard taxonomy, glossary, dictionary, is good for me and the industry.


                                                  A “taxonomy” and a “glossary” are not birds of the same feather. It is my understanding that a “taxonomy” is a classification scheme as in “Bloom’s Taxonomy”. Whereas a “glossary” is employed to document the definitions of the terms of a domain of discourse. At least this is my understanding.

                                                  As a systems engineer I always press for the establishment of a project glossary at the outset, but as Stephen mentions, that is a challenge.

                                                  I would like to suggest that a standard wiki (i.e., wikipedia) may not fully satisfy the requirements being expressed here, as I understand them. If we agree that the vocabulary of a domain of discourse possesses an architecture, then modeling may be the best manner to articulate and describe our vocabulary. The principle element of the model would be a “term”. Suggested attributes of a term ‘ might be its definition(s), its acronym [0..1], the terms status as the preferred term:Boolean, if not preferred then the identity of the preferred term, source citation, comment [optional]. Relationships amongst the terms adds additional understanding of the vocabulary. Uses of terms in a context can be represented. Just an idea to add to the discussion.

                                                  Modeling enables the capture of relationships between elements which may be more likely to satisfy requirements.


                                                  Thanks Geoff, a taxonomy is what we require in order to make material on this website more efficiently discoverable – a consistent classification of existing and new material.

                                                  The idea snowballed relatively quickly but if this leads to a collaborative effort to create a software testing vocabulary model, I’d be happy to help out wherever I could. I guess it would be relatively easy to derive a taxonomy from an established model but modeling a vocabulary will take a great deal longer (I assume, knowing little about data modelling).


                                                  @Stephen, seems crazy to me that you can have three (or more) interpretations of UAT – which effectively means that any of the terms listed above have multiple interpretations. Scary!

                                                  What do you think of Geoff’s idea regarding a vocab model for testing?


                                                  I am surprised no one has mentioned ISO-IEC 29119 Software and Systems Engineering — Software Testing Part 1 – Concepts & Definitions. The Contents section pretty much identifies the Testing through the SDLC you are trying to addres. Plus, as the name suggests, it also includes a Terms & Definitions section.
                                                  Why re-invent the wheel?

                                                  The drawback being you would need to buy the standards document to have visibility of its contents.


                                                  If I really understood how to model something it would be a good idea. Seems we are looking at vocab issue on the subject now.
                                                  Is it a taxonomy, a glossary, a dictionary, a Wiki, a Model or what? 🙂

                                                  As with many people, I use a Glossary in a lot of the docs I produce so people know what I mean when I use a particular term or acronym
                                                  The idea of a dictionary is really just an extended Glossary.
                                                  A Wiki takes that further, e.g. from a short one liner, to a short definition, to a short essay.

                                                  A Taxonomy, surely is one type of model, even if it’s a simple one.

                                                  I think each of these has value to someone at some point. Like having a full text book and a pocket guide on the same subject.
                                                  A Glossary is a good starting point. Thrash out all of the terms and short definitions.
                                                  This can then be expanded in two routes. 1 – A Wiki to take the definition and explanation further and 2- a Model to show when and how to use each term.
                                                  I think in Testing a Taxonomy is limited as it’s a bit 2 dimensional. A full model could be multi-dimensional showing all the contextual relationships and enabling the user to identify the appropriate terms and methods for each project and situation.

                                                  @Keith I’ve looked at ISO-IEC 29119, as far as you can without paying out, and at the moment I agree with a lot of the chatter on various social media sites, I don’t think it’s good for the industry. Saying that I’ve not seen the definitions provided by the standard.

                                                  To be honest, most of the standards I’ve seen, trained on or used are really best used to standardise the language used, attempting to ensure we all know what each other is talking about: But not for how to do the job. All standards I’ve worked with don’t go all the way and include all the terms used; you still end up with people talking a different language at times.

                                                  An initiative like is more likely to capture the vast range of terms used by all, and not just a select group of academics ( I do realise that some of the committee that defined the ISO-IEC 29119) are testing professionals as well).

                                                  Lets face it; the world today is a world of global collaboration and development by social media, not by committee and red brick university or institutional organisations.

                                                  Sorry, I seem to have slipped of subject.


                                                  @Paul, I concur that a taxonomy is required for discovery. Your original list confused me because from my perspective I recognized some things as classes of things whilst others appeared to be things. Other contributions were clearly directed towards a glossary vice a lexicon.

                                                  So then the question becomes how to group things. What is the basis of our classification scheme? Do we take a perspective aligned to the process classification schema employed by ISO/IEC 15288?

                                                  System Life Cycle Processes
                                                  Life Cycle Model Management
                                                  Infrastructure Management
                                                  Project Portfolio Management
                                                  Human Resource Management
                                                  Quality Management
                                                  Project Planning
                                                  Project Assessment and Control
                                                  Decision Management
                                                  Risk Management
                                                  Configuration Management
                                                  Information Management
                                                  Stakeholder Requirements Definition
                                                  Requirements Analysis
                                                  Architectural Design

                                                  ISO/IEC 15289 has an excellent (IMHO) classification schema for information items

                                                  Information Items
                                                  Concept of operations
                                                  Database design description
                                                  Interface description
                                                  Service catalog
                                                  Software architecture description
                                                  Software design description
                                                  Software unit description
                                                  System architecture description
                                                  System element description
                                                  Acceptance plan
                                                  Acquisition plan
                                                  Asset management plan
                                                  Audit plan
                                                  Capacity plan
                                                  Configuration management plan and policy
                                                  Development plan
                                                  Disposal plan
                                                  Documentation plan
                                                  Domain engineering plan
                                                  Improvement plan (process improvement plan, service improvement plan)
                                                  Information management plan
                                                  Information security plan
                                                  Installation plan
                                                  Integration plan (implementation plan)
                                                  Maintenance plan
                                                  Measurement plan
                                                  Project management plan
                                                  Quality management plan (quality assurance plan)
                                                  Release plan
                                                  Reuse plan
                                                  Risk management policy and plan
                                                  Service Availability and continuity plan
                                                  Service management plan
                                                  Training plan
                                                  Validation plan
                                                  Verification plan
                                                  Configuration management plan and policy (change management policy, release policy)
                                                  Improvement policy
                                                  Information security policy
                                                  Life cycle policy and procedure
                                                  Quality management policy and procedure
                                                  Risk management policy and plan
                                                  Audit procedure
                                                  Capacity management procedure
                                                  Configuration management procedure (change management procedure, release management procedure)
                                                  Complaint procedure
                                                  Implementation procedure
                                                  Incident management procedure
                                                  Life cycle policy and procedure
                                                  Maintenance procedure
                                                  Operational test procedure
                                                  Problem management procedure
                                                  Process assessment procedure
                                                  Qualification test procedure
                                                  Quality management policy and procedure
                                                  Software unit test procedure
                                                  Supplier management procedure
                                                  Supplier selection procedure
                                                  Training documentation
                                                  User documentation
                                                  Acceptance review and testing report
                                                  Audit acknowledgement report
                                                  Audit report
                                                  Configuration status report
                                                  Evaluation report
                                                  Incident report
                                                  Installation report
                                                  Integration and test report
                                                  Monitoring and control report
                                                  Problem report
                                                  Process improvement analysis report
                                                  Product need assessment
                                                  Progress report
                                                  Qualification test report
                                                  Review minutes
                                                  Service report
                                                  Software unit test report
                                                  User notification
                                                  Validation report
                                                  Verification report
                                                  Change request
                                                  Customer satisfaction survey
                                                  Request for proposal
                                                  Resource request
                                                  Risk action request
                                                  Service level agreement
                                                  Test specification
                                                  Process Specification
                                                  Stakeholder Requirements Specification
                                                  System Requirements Specification
                                                  Software Requirements Specification

                                                  Do we look to the OMG’s MetaObject Facility (MOF) and build a test domain business model?

                                                  The OMG’s UML Testing Profile provides some useful ideas on classification of test artefacts. Behavioral vs. structural. It is currently under revision. The OMG has also recently released the TestIF Specification (Test Information Interchange Format) into Beta. Another source for classification concepts.

                                                  Mention has been made of ISO/IEC 29119 which I think is a good starting point (IMHO) for a glossary as well as providing food for thought on building the lexicon.

                                                  I’d like to see the ISTQB’s glossary harmonized with the ISO and the OMG glossaries, perhaps there are others I’m not aware of that provide value. I believe standardization to be a good thing as it is necessary for effective communication. Standards are not necessarily without flaws. I feel strongly that the ISO does a disservice in charging license fees that individuals and small entities find to be a barrier. The fees are a clear inhibitor to use and adoption of the standards.: es exemplified in this thread.

                                                  As to “social media”, I question its ability to be as effective as a committee or any organized body bringing together committed individuals for the purpose of reaching a consensus to better a specific domain. Social media is too easily overwhelmed by bullies and crackpots. Apologies for the digression.


                                                  @Keith – thanks for the suggestion. I agree, if a body of work already exists and it’s something testers use, then well and good – let’s not undertake work for the sake of it. However, if we’re to use the ISO document and amend we run into copyright issues I’m sure?

                                                  @Stephen taxonomy|glossary|model|dictionary|lexicon – just Google each of those and you’ll come across several (albeit closely related) definitions of each – we could start with a glossary of terms for our new taxonomy or is it the other way around?!

                                                  @Geoff thanks for the further references, the list goes on :-). Gathering and harmonising existing glossaries might be the starting point?

                                                  A summary:

                                                  From what I can see, there is interest in (and a need for) a standard language for testing (that doesn’t cost 178 Swiss francs for everyone to access). There is interest in this being a community project but reluctance to allow the ‘squeaky wheels to get the most oil’ by social media being the basis for creating a language.

                                                  How about a community-elected committee of committed individuals who are prepared to take responsibility for taking this forward? A forum could continue to be a place where aspects of the project are discussed (proactively) by all and where progress and updates are shared. Ultimately it’s a committee who finalise definitions, concepts, relationships etc. Could this work and crucially, would people be willing to commit to what is a considerable body of work?


                                                  Knowing how much effort went into producing the ISO-IEC 29119 standard, I think 178 Swiss Francs is a small price to pay to avoid the need to re-invent the wheel & put in significant effort from multiple individuals to effectively re-write the standard from scratch.

                                                  Assuming everyone had access to the standard & wanted changes made, then I could act as liaison to get recommended changes reviewed for incorporation in subsequent revisions to the standard

                                                  A standard is not a document that everyone has to adheer to in its entirety, rather it is a series of guidelines & recommendations for what test professionals should consider in undertaking their role;

                                                  Accreditation to the standard is an option, but it is not mandatory (unless your customers require it of you)


                                                  Hi all, I wasn’t aware of this website until now:, I presume testers in the UK are aware of this. Are these standards (or part of) employed and if not why not?



                                                  I wasn’t aware of that site. Which is ironic that I am aware of Paul Gerrard who hosts the site.

                                                  I’m going to have to explore that site now.



                                                  I note the last dated update on that site that I could find was 2005.
                                                  It’s looks like a reasonably good source for info, but might be outdated a bit.

                                                Viewing 30 posts - 1 through 30 (of 47 total)
                                                • You must be logged in to reply to this topic.

                                                One Response to “Taxonomy of Software Testing Terms”

                                                New eBookMaintaining Quality Standards When Nothing Stays Constant
                                                Download Now