Commercial models beyond standard time and material, T&M or fixed price are being evaluated for their potential to drive differentiation and innovation amongst service delivery providers. Advanced commercial models are identified in the marketplace as outcome, unit, (service level agreements) SLA, and subscription based pricing and all can be applied against the variety of service delivery models. This is a topic being discussed regularly with industry analysts. Forrester says “outcome based pricing, OBP, has the potential for higher order benefits than just cost savings. Gartner says ” OBP this is the direction that mature outsourcing clients are taking in the current business environment.” SLA/SLO’s(objectives) are usually associated with a managed service/TCoE as the vendor must be able to control the testing (end-to-end) and also be able to greatly influence quality throughout the entire delivery process. With the concept and adoption of “testing-as-a-service, TaaS”, test factory model, outcome based pricing is being applied against services that were normally T&M only. For example you can contract with a service provider to create, maintain & execute automation. In this case “outcomes” could be number of scripts created and successfully executed/reported within a specific timeframe. You would also expect some quality measurement to be applied against these scripts as well to determine “outcome” attainment. In a subscription based model, you are signing up for a number of “units” for a set service(s) to be used in that month. Or sometimes subscription models are “all-you-can-eat”.
If this sounds interesting, here are some areas for you to consider before going down this path.
The maturity of your delivery for both the entire SDLC as well as testing needs to assessed/appraised. Ask yourself how often the team meets code-drop/delivery dates? What is the expected level of code quality upon delivery? Do environments/data delays occur frequently? Are the tests written well enough for someone to be handed them to automate? What software delivery tools are in play to keep the flow of information current and accurate? In any of the advanced commercial models, these deviations from plan will mount and be very costly. In T&M models, unproductive time is paid for by the client. In OBP’s there is no “unproductive” time. The order comes in or the cadence is set and we execute. If there are delays, client pays as the vendor has allocated the FTEs to get the job done.
Ask yourself if the team normally has a high defect rate after each release? Is it expected to have increased/excessive code drops that increase the number of times defect re-testing and regression get executed? There is no “carry-over” of unutilized units and yes you will pay more should go above your monthly units. I have seen non-functional testing like performance, security, cross-browser, and mobile testing start to go down the OBP path.
Budgeting for the variation is directly tied to the above maturity. In the case of a managed service, if the vendor actually beats the SLA/SLO, what will be the financial benefit to the vendor? You better have had that line-itemed in your budget. With the other OBP models, know your risk (impact & probability) and allocate money accordingly. Of course the SLA/SLOs, units, subscription base get adjusted periodically, but have some extra coin ready. You can always give it back after each month or store it up for that crazy release ahead.
If you do not have the data points to really quantify your current cost of quality, your risk is high as you are guessing at the impact & probability. My recommendation is wait to migrate towards OBP. If you are somewhere “in the middle”, look to get a formal assessment and implement a process improvement program to increase your operational performance against your results. Baseline & benchmark your current operational performance. You will need it to figure out what the future would cost based upon past performance.
Your delivery methodology (waterfall vs. agile/iterative) will need to be considered for obvious reasons. By the very nature, agile/iterative models creates constant waves of delivery and therefore the cadence and amount of testing is varied for each wave/sprint and don’t forget about regression. If you are not mature in agile delivery do not even to think about OBP.
Advanced commercial models can be attractive to some clients IFF they have a high level of operational maturity throughout the SDLC.
Paul Fratellone
Director QA Practice VirtusaPolaris
