- This topic has 13 replies, 11 voices, and was last updated 9 years, 2 months ago by
.
-
Posts
-
I’ve seen these terms used interchangeably before but I’m sure there are differences. What would you categorise as the main differences?
The origin is probably in IEEE825 from mid 1990’es. To some they are very fixed standardized documents with given order and context, to others it’s just a mind map.
Often I think of the Test Strategy as a High-level and inital test description, similar to what developers would write in a “High level design”. Later the Test plan details the test ideas, test data – similar to the “Detailed Design Document”. That would depend on the specific interpretation of the v-model
Also see the topic:
WOULD A BUSINESS MODEL CANVAS DO AS A TEST PLANIn ISTQB Terminology a Test Strategy is the description of the approach for testing on company level where the Test Plan is the application of the Test Strategy for a specific project.
The Test Strategy describes aspects like the standard Test Levels that are recognized within the Test Process, the Test Environments that will for used for each of these levels and how to use them (managing Test Data, deployment of new release), the interaction between the development process and the test process, differentiation of the test approach depending on risk, the tasks to be executed and deliverables to delivered, tools to be used, metrics to be followed and reports to be written.
That’s about the same content as a Test Plan. Therefor it is said that a Test Strategy is a blueprint for the Test Plan.
Where the test plan describes the organisation of the project, the Test Strategy describes the organisation of the company and how testing fits into this organisation. Where the Test Plan describes the specific change within the context of the project, the Test Strategy references to the whoel company context (applications/systems and their interactions, …). And so on.
With regard to the Test Plan, the distinction can be made between the Master Test Plan and the Level Test Plan (IEEE 829 – 2008). The Master Test Plan covers all test levels for a project. For each Test Level for which it is relevant, a Level Test Plan can be made.
E.g.
the Master Test Plan will document the test milestones for the project; the Level Test Plan more refined planning dates for specific tasks within a Test Level.
The Master Test Plan will indicate in which period certain roles will be needed. The Level Test Plan will assign names to these role.So a Test Plan could contain a section in relation to a Test Strategy although they are fundamentally different things.
Think of it this way:
You are fighting a battle and you PLAN to move your squadrons (TEST RESOURCES) on to the battle field tomorrow (TEST SCHEDULE). They will be armed with guns (TEST TOOLS) and tanks (AUTOMATION TOOLS) with which to shoot at the enemy (DEFECTS). However, your STRATEGY is to wait until the first wave of the enemy moves in (CYCLE 1 of your TEST SCHEDULE) before manoeuvring your armoured division, round the side in a flanking manoeuvre to kill off any stragglers and inflict maximum damage (REGRESSION TESTING).
Maybe this helps?
So for me the strategy deals with the how and the plan deals with the what, the who and the when.
Does that metter? Strategi of how to aproachn testing should be, and is, part of a test plan, as the product may differ from each other. Each product demands overview of strategy, testing levels, commitment level of PO, etc. So, test plan should and it does include the complete strategy, as it contains testing activities that are giving us broader picture of strategy we apply.
In simple terms, strategy is about the ‘what’ and the ‘why’ whereas planning is focussed on the ‘how’ and the ‘where’. I suppose that makes it true that strategy is about the bigger picture and planning is filling in the detail to achieve that picture. It’s impossible to plan successfully without understanding what the goal is and why you want to get there. A good test strategy needs to take into consideration the overall strategy of both the project, the product or the overriding business. It is absolutely key to get this right and in sync. Once this is done, the plan can follow what ever methodology has been defined as suitable. They need to be treated separately and in the right order. Like love and marriage, you can’t have one without the other!
Chris Ambler
I think Chris and Stuart have come close to it. In answer to Edin, yes I think it does matter. If you don’t address the issue of strategy in advance of planning you are likely to always be ill-prepared and (especially in a multi-team environment) liable to confusion as to how to approach test planning and, ultimately, testing. To quote an old military maxim, ‘train hard, fight easy’ or, as it is sometimes put, testing should be ‘90% preparation and 10% perspiration’, not the other way around. In terms of Kipling’s ‘6 honest serving men’, strategy is about the ‘how’ and the ‘why’ whilst planning is about the ‘what, when, where and who’ (i.e. scope, schedule and resource). I think confusion arises because in strategy you have to make reference to ‘why and how’ you should decide on the ‘what, when, where and who’ when planning. If it helps, think of it like the difference between and class and an instance of an object; the class (strategy) provides an infrastructure on which you can build your instance (plan) either directly or, if you need to, you can adapt the infrastructure through inheritance. Having said all that, let me share my experiences in this area…
In my last role (with a major UK software company) I spent several years (2005-2013) wrestling with this issue. As ‘Principal Test Analyst’ then ‘Test Architect’, my main responsibility was supposed to be to direct test strategy across multiple development and test teams for an integrated product suite, so that the teams could then draft their own test plans in harmony. The problem was, nobody seemed to understand the difference between strategy and planning. In the many documents I produced I came up with a variety of definitions (in addition to the one above) to try to resolve the strategy/plan confusion. As an ex-officer of the RAF, I also tried Stuart’s military analogy and really thought I had it cracked with this definition in 2008:
Strategy, Tactics and Plans
“The Concise Oxford Dictionary tells us that the word strategy originates from the Greek for ‘generalship’ and is defined as ‘The art of war’. More specifically, it refers to manoeuvring forces within a theatre of war to gain an advantage. The general must know what forces will be required, how to prepare them, where to deploy them and, most importantly, when and where to engage the enemy in battle.
The word tactics derives from the Greek for ‘to set in order’ and is defined as ‘The art of battle’, of how to defeat the enemy on the battlefield. The tactical commander is most likely to achieve victory if the strategy of the commander-in-chief has given him some advantage over his opponent. Who actually wins the battle depends on decisions made by the tactical commanders on the battlefield.
The analogy is simple. Test Strategy refers to the preparations we make so that we have an advantage when it comes to deciding how to test our application for a specific release. Test Plan refers to the actual decisions we make about how we test our application for a specific release. Test strategy gives us an advantage because it increases our options for testing and provides guidance on how to select the best options; it thereby facilitates test planning and the implementation of our testing but does not dictate either, the final decisions are left to those entrusted with the testing. Having a good strategy, therefore, does not guarantee success but it can reduce risk and significantly increase our chances of success.”I followed this up with sections on applying ‘best practice in current context’ and how test strategy would pave the way by for planning by ensuring the teams would be equipped with appropriate knowledge of the AUT, test methods, tools, skills, organisation, risk management and lines of communication. I supported this with such things as procedures for teams to coordinate and collate testing before and after integration. All this was in vain, some teams still expected me to provide them with a test plan for each release/iteration (we ‘went Agile’ in the midst of this) whilst at other times I had some of the Product Development Managers telling me I was not ‘to tell their teams how to test’. In 2011, after 2 more attempts at documenting a strategy and in desperation, I gave up on the documentation and went for a full-blown ‘Agile’ approach. I spent several weeks meeting and consulting with all the major stakeholders (aka the Management Team) the team leaders and their teams (i.e. my ‘customers’) and asked them what they expected from the Test Strategy. I was given a list of the issues they had related to test planning and testing. I consolidated these into a prioritised ‘Test Issue Backlog’ and I presented the Management Team with my new ‘Strategy for Continual Improvement in Testing’, which was to address this backlog of items in priority order (adding new items as they arose) working in consultation with the teams. I was promptly told that this was not good enough and they expected the Test Strategy to be ‘fully documented’. Aaaarrrghh!
After another 2 years of preparing ‘Strategy’ documents (which sometimes bordered on Test Plans and which were almost always ignored) there was a consolidation of roles across the group of companies. As the only ‘Test strategist’ in the Group, my role (and I) became redundant. Read into that what you will!I think the description at guru99 describes it quite well.
http://www.guru99.com/test-plan-v-s-test-strategy.htmlWhat are people’s thoughts on strategy in an ‘Agile’ environment? When I mentioned in my previous post that some Product Development Managers had said I was not to ‘tell their teams how to test’, I had some (but not a lot) of sympathy for this view. Their argument was that, if we insisted Agile teams comply with a specific test strategy, we were imposing restrictions on how they worked and this could prevent them from attaining their goals (or at least give them an excuse for not attaining their goals). Of course, the same PDMs had no problem with the programmers having to use the same programming language, IDE and coding standards to implement development in according with an architecture dictated by the Software Architects. We did resolve this issue (I’ll spare you the detail) but where did it originate from? Does the idea of over-arching strategy really go against the concept of Agile or (going off topic) is there still a problem of attitude towards testing i.e. that testing is not a ‘discipline’ in the same sense that analysis, design and programming are?
I like the idea that stategy is the how we need to test.
Created looking to the business risk and cross to the quality characteristics and try to find what is the most important to test. As we know test 100% is impossible so we need a strategy to get our target. So we need to create the staryegy before the Test Plan.
Test Plan to me is like a contract that we need to follow. We define the terms, the condicions and everything we will need “to the arrow find the target”.@Andrew, Test STRATEGY and Agile don’t mix very well because of the Agile ideas.
Test STANDARDS however should not be a problem.
I totally agree with the notion that you should not tell an Agile team how to test. I believe in professionals doing their job.
A professional however is aware of the (company) standards for testing just as a professional is aware of the coding standards.
So in your case I would have rewritten my strategy in such a way that it would become a testing standard at the same level as the coding standard.
Now the only thing you have to say is: please comply with the current standard – without telling how they should comply, that is their perogative.@Kasper, I totally agree with the sentiment that Strategy and Agile don’t mix very well, also with ‘professionals doing their job’. With regard to your suggestion about ‘standards’, the company policy at the time was that each product team (the body of people responsible for development of a product, of which more in a minute) should be a self-sustaining business entity, so, ironically, the product team were given total freedom as to their development practices. Ergo, there were no company standards for testing as such; it was my responsibility to set testing standards for the larger product team, via the strategy and via governance (I shun the word ‘policing’) of testing practices.
I was being a bit naughty in my earlier post; in the interests of fuelling the debate, I didn’t put the question in the full context. Where you have a single Agile team working in splendid isolation, I would agree that Strategy and Test planning should be combined (in so far as company policy/standards, sometimes called strategy, permit) and the sole responsibility of the Agile Team. In this case, however, we were implementing scaled Scrum, with 10 development teams working in simultaneous iterations. Each team worked on a separate module of the software but the modules could be integrated and used in various combinations. At the end of each iteration, the separate branches of code were merged and passed to a central test team for ‘system’ testing. In this situation, it was imperative that there was selective integration testing during each iteration and that required a strategy that covered not only how this should be done but also who should take responsibility for testing the areas of integration.In summary:
• If an Agile team is working in isolation, then there is no need to differentiate between strategy and plan.
• Where development is ‘scaled’ over multiple teams working on separate development threads of an integrated product, there has to be a strategy to guide and coordinate the testing between the teams.
• The more complex the integration issue, the more prescriptive the strategy has to be and the more it begins to encroach into what would normally be considered to be planning.I would say that the same principles apply to non-Agile methodologies but the concept of a strategy is, as you say, more at odds with Agile.
@andygorman , I have handled a similar situation in the past by settting up a seperate intergration Scrum team with the sole responsibility of systems integration and integration testing. There was one huge difference: coding and testing standards were in place,
The integration team could not say to the different development teams what to to, but they had complete access to the teams.
In this way they knew what was coming to them, what was and what was not build / tested properly and they could report to the product owner about compliance to coding standards and test standards.
The product owner then knew what to do 😈
- You must be logged in to reply to this topic.