In this article on several different test approaches for Manual Regression Testing and how they compare to each other, I focus on Regression Testing rather than general Test Execution in the SDLC.
This is because I believe we need to consider extra factors such as repeatability, transferability etc. to select a Regression Test approach and different approaches can be adopted under different circumstances.
I haven’t considered Automated Testing in this article, even though I’m a big fan of it. This is because I’ve found that project teams often don’t have enough time/skillsets/budget to build automated tests or regression tests aren’t worth being automated because they’re not always cost effective. Thus, we often need to build and run a Regression Test Suite with a manual approach.
When I started thinking about this, I asked:
- What kind of approaches are there? And what are the pros and cons for each approach?
- How can we choose the best approach under different circumstances?
- Some testers ‘HATE’ Script Based Testing. Is Script Based Testing totally useless?
I’ve used three manual testing approaches – traditional Script Based Testing, Session Based Testing and Checklist Based Testing in my two years’ testing experience and created the following diagram based on this.
Session Based Testing is a popular test approach, especially for ‘modernised’ testers. It’s a very powerful and effective testing approach with low maintenance. But it can take a lot of time to ‘learn’ about the product unless testers already have comprehensive domain knowledge. This is not ideal because regression tests are usually expected to be run in a short period of time just to make sure that no regression has happened. Also, Session Based Testing doesn’t usually produce detailed artifacts such as Test Scripts and this makes the repeatability difficult.
<Session Note Example>
In my current team, we’ve been using Checklist Based Testing as the main Regression Testing approach. Checklists don’t require as much maintenance as Test Scripts do, but still give enough guidance about what to test and how to test. Checklist Based Tests still require a reasonable level of background knowledge about the product for testers. Thus, there is a trade-off between the maintenance effort required and existing knowledge/learning time required to perform the test execution.
I’ve used the Script Based Testing approach once, where I wrote and used test scripts for Feature Testing as my client requested. But, after the product went live, my scripts weren’t maintained and now they’re not used at all – a shame as I spent a lot of time and effort on them! Not that I want to advocate Script Based Test, especially when I’ve seen that the approach failed in the long term.
I believe that, in some circumstances, Script Based Testing can be the right choice for Regression Testing. For example, if testers without enough background knowledge are often assigned to Regression Testing and there’s not enough time for them to learn about the product – and there’s a very low risk of regression – Test Scripts will provide more useful information and guidance to testers than Checklists or Session Notes. Of course, Test Scripts are useful only when they are up-to-date. But considering that maintaining Test Scripts would normally take less time and effort compared to knowledge transfer – plus human-based knowledge transfer doesn’t work if people with the knowledge leave the company – regular updates of Test Scripts can be a worthwhile investment.
<Test Script Example>
If there’s a test approach with all the benefits of those three approaches – low maintenance, high repeatability and low requirement for existing knowledge – it would be ideal. I haven’t come across one myself yet. Until I find one, I’ll decide on my Regression Test approach by considering the pros and cons in the diagram above and the project circumstances.