August 22, 2014 at 2:45 pm #3588
@Keith – I guess it’s a relatively small sum of money but my point was that it appears people are not prepared to spend it in order to access the standards. I doubt the sum is the sole barrier to testers and organisations using these standards?August 22, 2014 at 2:47 pm #3589
@Stephen, Stuart Reid was involved in both the original British standards and the ISO 29119 standards. I’m going to get in touch with him to ask for his views on this conversation.August 22, 2014 at 2:57 pm #3590
Stuart actually presented a webinar on ISO 29119 last year: View it hereAugust 27, 2014 at 12:12 pm #3698ErkkiParticipant@erkki
A taxonomy works best the items in it could be considered as coordinates for nuggets of useful material, so that any single item could have few coordinate values in place, and then same item could be listed on all relevant places where the material is relevant. Hardest would be the artificial constraint where the taxonomy is assumed to be a single hierarchy and that each material would be relevant on only one point of hierarchy / locatable via only one dimension only. For example there could be checklists for social aspects of test planning: they have both testing life-cycle releveance, relevance to “Test management” and also in “People” dimension. As there really (and fortunately) is no single testing approach guiding we all to assume certain hiearchy in our heads, so there also is no single dimension to have all relevant classifications in it as one value, but more like items float in multi-dimensional space and several coordinates are potential ways to find the material.
It looks like the Non-funtional testing is a subset of Acceptance testing (a simple copy paste error?). I’d suggest to adjust the upper organisation as follows:
Testing through the SDLC (avoiding acronymes would be better here, I suppose, like “Testing through the software life-cycle”)
(Each waterfall phase / test level listed here, from Requirements to Maintenance)
Then from testing methods onward it seems mostly logical, except 2 things:
1. Only place I find questionable is the “results analysis” alone with no context. This points out the missing of the lack of plan-driven testing continuum, or testing life-cycle. Testing life-cycle in all its artificial simplicity is great for organising different testing practices (methods / tools / techniques). For example the TMap had these 5 that encompass useful concepts and logical continuum of important testing activities and decisions — as long nobody assumes they follow each other linearly or that all should be put into some document 🙂
Planning & Control
2. The division of testing techniques and methods is IMHO somewhat artificial and they might be grouped as “Testing practices”.
BTW, if the proposed changes in all this thread posts have been implemented somewhere else than the original post, how about giving the link on this thread?
Cheers, ErkkiSeptember 1, 2014 at 7:49 am #3774RainerParticipant@rdeussen
Some more classifications of testing methods or kinds of testing (might overlap with threads shown above)
ad hoc testing
basis path testing
best representative testing
boundary value testing
branch condition combination testing
build verification testing
business process-based testing
component integration testing
condition determination testing
context driven testing
control flow based testing
data driven testing
data flow testing
data integrity testing
database integrity testing
decision condition testing
decision table testing
equivalence class testing
evaluation based testing
extreme value testing
feature or function integration testing
finite state testing
follow up testing
glass box testing
gray box testing
integration testing in the large
integration testing in the small
keyword driven testing
long sequence testing
modified condition decision testing
modified multiple condition testing
multiple condition testing
operational profile testing
resource utilization testing
site acceptance testing
state transition testing
subject-matter expert testing
system integration testing
unit integration testing
use case testing
user acceptance testing
user scenario testing
white-box testingSeptember 1, 2014 at 12:44 pm #3786
Hi Erkki, thanks for your input! Can I summarise your suggestions as follows?
1. Testing through the sofware life-cycle
Testing (as a lifecycle phase)
2. Testing Types
3. Test Practices
to include Methods and Techniques as above
4. Test Process as above
5. Test Automation as above
6. Testing Artefacts as above
7. People as above
Seven overarching areas like this would be a great starting point for getting a taxonomy that works on this platform but if we can apply relevant tags to all content resources, then the idea of material being discoverable by different coordinates becomes possible.September 2, 2014 at 8:05 am #3793
Thanks Rainer, 159 terms I count there. I wonder where they would sit in a taxonomy as opposed to the alphabetical list? I also wonder how many more terms are commonly used? Have we come to a near exhaustive list?
I’d also be interested to get the thoughts of the few hundred people following this thread but so far not joining the conversation too 🙂September 2, 2014 at 2:07 pm #3822GeoffParticipant@gashuebrook
@Paul, Erikki’s puts forth a schema that is reasonably harmonious with ISO concepts for product (i.e., systems and software) development and reasonably harmonious with the concept of a taxonomic hierarchy where one employes generalization and specialization. Your thread has surfaced the fact that a taxonomic schema isn’t universally understood for its role in a vocabulary of a domain discourse.
Even in the light of the recent turmoil and dissension regarding ISO/IEC 29119 by key principles in the Test community, the concepts of classification developed over the years by ISO/IEC should not be ignored. IMHO
ISO 15288 “Life Cycle” “stages” of a system = Concept, Development, Production, Utilization, Support, Retirement
During the “Development” stage the following technical transformation processes may be executed = Stakeholder Requirements Definition, Requirements Analysis, Architectural Design, Implementation, Integration, Verification, Transition, Validation
During the “Development” stage the following project governance processes may be executed = Planning, Assessment and Control, Decision Management, Risk Management, Configuration Management, Information Management, Measurement
During the “Development” stage the following project-enabling processes may be executed = infrastructure management, human resource management, quality management.
I tend to place “processes” as a central concept in my thinking. Why? Firstly, because I am a Systems Engineer. Secondly, because it is through the execution of a process that transformation occurs. It is how value is created. Processes define “What”, they do not define “How”. “How” is defined in the lower elements of the behavioral hierarchy by process activty, tasks, methods, and procedures. Next, I have structure (e.g., specifications, architecture descriptions, designs, reports, roles, etc…) as these things are inputs and outputs of processes (i.e., technical transformation, project governance and project-enabling). This is how I might start at building a taxonomy. But this is what I am comfortable with, it is how I think. It is how I have been conditioned to think of my domain, mostly through my exposure to ISO/IEC standards.
This figure is an attempt to illustrate how I think about a process and its elements . This view of the model is not complete, it is merely one perspective of my process concept model.
Hopefully one perceives the harmony with classification ideas put forth by Erikki. They are not equivalent, but they are not discordant either. “Test Artifacts” are structural elements. “People” relate to Roles. “1. Testing through the software life-cycle” relates to processes in general. Methods relate to @Erikki’s concepts of testing practices and behavioral aspects of testing. The “How” of a process.
@Paul, you state in your reply to @Erikki “discoverable by different coordinates” which suggests, to me, that an entity might be discoverable by way of more than following a hierarchical classification tree. Was this your intent? If it was, then aren’t you really speaking of an ontology now? A much more complex concept, at least to my understanding.September 8, 2014 at 9:55 am #3900
Geoff, an ontology is a stretch too far I think! What I meant was that a piece of material would be discoverable in more than one way – it would be found at relevant points in the taxonomy and not just one.September 9, 2014 at 11:08 am #3957
Given the interest here in the topic of a standard language, it would be good to get your thoughts on the ISO 29119 standards – Ronan has started a discussion on this. Why are some testers so vehemently opposed to this while others are ‘ok’ with the development of standards?February 10, 2015 at 8:50 am #6690ClaudiuParticipant@claudiu-draghia
I have been working in the past year or so on a taxonomy in regards to software testing information but I believe it can be applied in general for software testing.
It is is a visual shape: The Testing Map
If you consider it useful I can also provide you the whole tree structure. There are around 300 areas on the map.February 18, 2015 at 9:08 pm #6838Adam HepnerParticipant@adam-hepner
I have a quiet proposition: this form of discussion makes me feel a bit… dizzy – it’s hard to follow, does not let itself be linked properly, and change propositions are not really properly managed. The developers have perfect tool for managing collaborative discussion and changes to text documents. It’s obviously GitHub :). I’ve created a quick repository here: https://github.com/AdamHepner/taxonomy-of-software-testing-terms (it is under creative commons, and I will initiate it with content from this discussion in the coming days). I encourage everyone to clone the repo, provide additional explanations, and submit merge requests and/or issues. Together we might make it one of the best repositories of the knowledge in the web!February 23, 2015 at 8:38 am #6884
This is a great idea Adam and you are right, the conversation has many tangents. I look forward to the next version.March 15, 2016 at 12:30 pm #11171Ronan HealyKeymaster@ronan
I thoughts this was an interesting article by Donald Firesmith on taxonomy of software testing. He has divided it up into a number of different parts.March 19, 2016 at 11:49 am #11188Jeba QptParticipant@jebaqpt
It involves different phases such as Initial/Planning phase, Requirement analysis phase, Design phase, Coding phase, Testing phase, Delivery and Maintenance phase.
Initial Phase involves gathering requirements by interacting with the Customer.
Requirement analysis phase involves doing detailed study of the customer requirements and judging possibilities and scope of the requirements.
SRS (System Requirement Specification) will be created in this phase
Design phase involves dividing the whole project into modules and sub-modules by doing High level Designing (H.L.D) and Low level Designing (L.L.D).
Coding Phase involves creating source code or program by the programmers by referring the design document.
Testing Phase involves getting clarification for the unclear requirements and then writing test cases by Testing Team based on the requirements.
Delivery & Maintenance phase involves installing the application in the customer place and providing the details such as release notes to the customer.
Read and learn about http://qtpbook.com/2011/08/13/software-testing-terms/March 19, 2016 at 4:01 pm #11191GeoffParticipant@gashuebrook
When I first read Donald’s paper, his taxonomy didn’t sit well with me. Flattening the multiple dimensions the way the taxonomy does just doesn’t seem to be the right way to conceptualize test. Each to his own.
The contributions of Jeba and several others are excellent examples of the challenges of this proposal. ISO standards have already established system, software and test vocabularies, yet it is clear from several terms used by others in this thread that these international standards are not having the influence they should be having. I suspect it is mostly due to the “pay to play” business model that ISO, IEC, and IEEE observe. The expense of accessing these standards is prohibitive for most. While I do not assert ISO standards to be the “Holy Grail”, they have a place in the ecosystem.
We are all entitled to our opinion, but whether that opinion is acknowledged as having relevance and value by our peers is another matter.
Ronan, Has there been any real movement on your proposal?May 9, 2016 at 10:21 am #11757
This discussion petered out somewhat I guess – it’s pretty daunting but I think @stevean’s suggestion is a good starting point:
A Glossary is a good starting point. Thrash out all of the terms and short definitions.
This can then be expanded in two routes. 1 – A Wiki to take the definition and explanation further and 2- a Model to show when and how to use each term.
I think in Testing a Taxonomy is limited as it’s a bit 2 dimensional. A full model could be multi-dimensional showing all the contextual relationships and enabling the user to identify the appropriate terms and methods for each project and situation.
If there’s genuine interest in this being a member-driven project a glossary of terms with short definitions sounds like a good place to start. Any volunteers to get us started?
We could start by having terms defined in this thread and then branching the conversation into different threads as it grows?
Open to other suggestions of course.
- You must be logged in to reply to this topic.