There’s just too much talk about the cost of testing says Rodger Perkins. What we should be focusing on is value and the outcomes...
Testing costs too much. How many times have we heard this from our sponsors or stakeholders? Or as a test sponsor or stakeholder, how many times have you said this to your test team? While the conversation is about cost, it's one that will go nowhere and usually result in a reduction of test scope and/or activity. While this potential reduction in test activity doesn’t necessarily lead to a lower level of quality, it will always result in less coverage, leading to lower confidence. I believe that a major outcome of the test programme is confidence in the product and confidence to take the product to the next stage or phase. Providing confidence in outcomes is a key outcome for all our engagements. Having a suitable budget to enable this confidence is critical, but talking about cost alone will not see budget flowing your way.
As test professionals, we need to change the discussion from a cost-based activity to a value and outcome-based activity. Sure, there is a cost involved in everything – but the real focus should always be on value. As a sponsor, what do you get? As the tester, what are you going to provide? What are you giving? Historically, testers are not great at talking about the value of the activity that they complete, and this needs to change. While not testing won’t reduce the quality (testing doesn’t create quality or failure), it can definitely improve the quality of the outcome. Testing is not limited to software development projects, it happens in every development or production activity.
Each and every one of us completes test cases on a daily basis as part of our normal routine. The coverage of testing should be determined by the risk of failure, but also by the level of desired quality. How we fund coverage needs to be based on valued outcomes. We need to value our work and demonstrate value to our sponsors.
To change the discussion and move away from a focus on cost and replace it with a discussion on value, requires a different way of thinking about what we do.
In a value-based discussion, we need to talk about what testing delivers, what are the outcomes and what the sponsor will 'get'. Generally speaking, we enjoy the concept of having something more than for paying for it. This is why we should talk to our sponsors about what they are getting. We can use visual management boards to show our sponsor what we are doing as they help make our largely invisible work visible. An estimate generally includes hours and often costs for a variety of testing tasks. But very rarely is an estimate supported by a statement of value. Rather than simply focusing on hours and tasks, I would like to see testers articulating value statements. For example, we know that a test plan will be written and roughly how long it will take. So merely stating this within an estimate is easy, but also slightly useless at the same time. What would be useful would be to replace a standard estimate with a statement of value – why is the test plan being produced and what will the outcome be? What does the sponsor get from the test plan?
To change the approach, we need to understand what the sponsor is looking for, but also – most importantly – what the sponsor values. There is little point providing a sponsor with a car to get from A-to-B if he or she values walking. I was once able to confirm the value of producing a test plan at $135,000. I was surprised by this as the test plan was 48 pages long, and I remember thinking that I could have bought a really nice car instead. But the value wasn’t in the 48-page document in its own right. The value lay in the weeks of discussions that had taken place within the project team – all those discussions that had enabled the test plan to be produced in the first place.
So we have two options when talking to our sponsors about testing:
1. Telling a sponsor that it will cost $135,000 to produce the test plan will probably end with a series of mainly difficult questions.
2. Telling the sponsor that they’ll get a clear understanding of the test project, that members of the project team will be involved, development and implementation issues will be identified and resolved, missing requirements will be identified, discussed and corrected... it just all sounds so much better.
Of course, option 2 sounds great, interesting and inclusive. It paints a picture of an outcome, giving the sponsor something to value. It doesn’t mention the amount, even though I know that they’ll ask for the cost at some point. This way, the sponsor will believe that they are getting something valuable and that they – and the rest of the project team – are part of the process.
If you are a tester, I encourage you to stop talking dollars and cents. Be aware of the cost, of course, but focus most on outcomes.
While testing can be expensive, I don't believe it costs too much. The cost of testing is what it is. We should stop thinking about it and focus on the value provided. Testing needs to support business delivery outcomes. Good testing will always support the solution and the business. Using testers with a solid knowledge of test practice, supported by partnerships and relationships that enable smarter testing to take place, will give you confidence and value.
In my next article, I'll talk about how to reduce the cost of testing while continuing to focus on the value being provided. I'll focus on how Assurity can help by reducing rework, eliminating waste and using new test techniques to improve efficiencies, effectiveness and coverage. I'll talk in more detail about how visual management boards help to demonstrate value, track progress and communicate achievements. Improving value can be achieved in various ways – and often, the easiest way to improve value, means simply to be better.