QA estimation for switching to new operating system

[Tassawer]

If you have a mature product which is extensively tested on a specific operating system and now management decides to provide support for a new OS.

How much issues you can expect to have introduced due to this change in operating system ???

[Roman]

I advise not to promise any plain estimations without “pilot test run”.

Use something like context-driving school recommends (James Bach, Michael Bolton), add to it also approach advised in James Whittacker “10 minute test plan” (there is also blog post about it)

1. Refuse to provide estimate until you do some initial “pilot” test run (“pilot” like “pilot project”). It can go smooth, it can run into critical system failures from the start. Promise to provide estimate after pilot project successful run, only if the SUT be testable enough. You can’t test SUT with OS if SUT basic capabilities don’t work, like it refuses to start at all right ?
2. Create a plan for the pilot test run with some crucial points / capabilities.  Estimate how long it will take. Don’t be too optimistic, use 1.5 multiplier as you don’t know yet what issues are there and how many questions you’d have to ask.
3. Do the pilot run
4. If you run into very critical failures or blockers fixes to which are hard to estimate, still refuse to give testing estimate. Refuse until pilot run be judged successful.
5. After you find that you can run most critical capabilities more or less successfully, knowing what issues you found with the last run, you can finally give more accurate estimate.

Stand your ground, use context-driven arguments:

1. “I can provide accurate estimate for testing when having no accurate estimate for issues there. If you are not ready to tell me what issues are there, I cannot provide any accurate estimate”.
2. — Testing will be finished when all the critical bugs get fixed. Can you tell me now when you are going to fix all critical bugs?
   — But we don’t know what bugs are there.
   — By now testing team doesn’t know it too.

[Darwin]

Sometimes I had issues with running the same script on both Windows 8 & 10. Then later found out, it was because of the environmental setup. So if you are using Java I would suggest to keep an eye on importing the jar files and libraries.

[Author]

Posts