Following his presentation at CAST 2013 in August, in Part 2 of his thought piece on mind mapping, Aaron describes how a map is worth a thousand words...
In Part 1, I described what my values are as a tester and how that drives my approach to communication and testing away from massive pre-scribed tests and onerous documentation towards a leaner, more transparent approach. So what is this approach?
I like to employ a style of software testing that emphasises the personal freedom and responsibility of the individual tester to continually optimise the quality of his/her work by treating test-related learning, test design, test execution and test result interpretation as mutually supportive activities that run in parallel throughout the project. When performed by a skilled tester, this approach yields valuable and consistent results.
This approach is comprised of three elements: The Heuristic Test Strategy Model, aspects of Session-based Test Management, presented in a visual model using mind-mapping software. This ensures that testing is performed systematically and intelligently and be auditable, accountable and traceable.
Heuristic Test Strategy Model
The Heuristic Test Strategy Model (HTSM) by James Bach is a set of patterns and heuristics designed to spark ideas around developing a test strategy. It is available to download for free from James’ site. I was fortunate to discover the HTSM early on in my testing career and I can say without any doubt that it is the most useful tool I have ever used. I even carry a copy with me to client sites.
For the purposes of creating a Visual Testing Coverage Model (VTCM), I want to investigate and map out all the different aspects of the product, as well as all the ways in which a stakeholder may value the product. For this reason, I use the Product Elements and Quality Criteria categories for the VTCM.
Session-based Test Management
Session-based Test Management (SBTM) is a way to structure exploratory testing and offers a framework for reporting and measuring testing activities.
The important elements of SBTM are:
Charter: This briefly describes the agenda or mission of the testing session.
Time-boxed session: This is an uninterrupted period of time spent testing or pursuing the Charter. Session lengths are typically 60 to 120 minutes, but you can timebox in half or full-days at the expense of granularity of measurement.
Session Report: This reports what occurred during the test session and can include:
Charter or mission
Start and end times
Notes on how testing was conducted
List of any bugs found
List of any issues encountered
Time spent off charter (called ‘opportunity’ in SBTM)
If necessary, percentage of time spent testing, time spent investigating bugs, time spent reporting bugs and time spent setting up data and test environments.
The kind of information recorded can, and should be, tailored for the project. If a standardised reporting format is used, the reports can be parsed to aggregate data that is deemed useful. There are also many tools that support SBTM.
The advantage of SBTM is that a tester has the freedom to apply their skills to actively investigate the product, while still being accountable and their work traceable. The advantage over pre-scripted testing is that the tester is reporting on what they actually did, as opposed to describing what they intended to do at some point in the past.
In practice, scripts are often obsolete by the time the software is delivered and, even under scripted conditions, testers will deviate from the script and two different testers may interpret the script two different ways. If important information is discovered under these deviations, this information is usually lost. SBTM allows rapid execution of tests and gives the tester the freedom to perform the next best action based on the information they have just learned.
Find out more about SBTM here:
The testing process is not a nice, neat, linear process. Information comes to us non-linearly and often spontaneously. I need a way to capture this information as it arrives without being concerned about how to adjust it to a linear model (such as a test plan document written in MS Word). I also need a way to communicate non-linear, complex information in a way that is easy to understand and process. So far, I have found that mind-mapping software is the best-suited tool for these goals. Along the way, I have discovered a wonderful side effect of using mind-mapping software: unsolicited feedback... which I will go into later.
So, how do we begin?
1. Start planning the framework
I like to begin by arranging the elements of the Structure and Quality criteria around a central node in the mind-mapping software. I then identify the properties I’m interested in, or may be interested in (after all, in these early days, who knows what may be important and the point is to stimulate your thinking in ways you may not have considered) and begin the framework for your map:
2. Start learning and collecting information
Before setting sail for uncharted lands, we should gather as much information as we can. After all, we are in a state of ignorance about what we will encounter. There may be tales of great mountains, of bountiful oceans and of savage beasts. There may be scrolls written by soothsayers and there will be accounts written by previous travellers.
In other words, there may be specification documents, responses to RFP’s, wiki pages, high-level descriptions, requirements documents. There may be project managers who can tell us great tales and business analysts with promises of bounty. We should begin mapping now so we have a frame of reference for our journey, but also fully expect that we will change it, add details, remove wrong details etc when we get in there.
The act of mapping sparks new questions allowing you to ask more focused questions of the people around you. The map becomes the tool to make the map better. This is the beauty of an organic system. Give it some initial conditions (The Heuristic Test Strategy Model, for example) and it almost builds itself. Here is the same map after a conversation with a developer:
New parts have been added, we've learned a bit about the structure and the platforms and what is required to install it all. We were able to ask good questions because we could see where the silhouettes were that needed to be illuminated.
3. Begin touring
We feel we have a useful framework to begin the actual work of touring the product. We know where the general inlets are and where the major landmarks are. We have enough to make landfall. Now we get the lay of the land. "Oh, here's a river we didn't know about. Better map that so we can take a closer look later." "This mountain is a surprise. I wonder if anyone knows about it".
As an example, as I installed something, it asked me for a port number . “Interesting,” I thought. “I’d better map that as a test idea”:
Eventually your map will be fleshed out and you’ll be confident that it is a pretty good and useful model of the software under test.
4. Create test charters
With a little tweaking, you've almost automatically generated your test charters from your visual model and you can attach your test session reports directly to the model.
5. Keep adjusting the map
Your map is a model. As your model changes, so will your map. Parts that you thought were important will be deleted and parts you didn't know existed will rise to prominence. The map is a reflection of YOUR understanding of the product at a given time.
It also has other uses. You can use it to manage the testing process by assigning testers to a branch, letting them take ownership of all sub-branches. This lets you quickly assign work without having to go through requirements and specifications one-by-one and manually assigning individual tasks. You could colour branches that you're currently working on and shade in branches that have been completed. At a glance, you can see what has been done, what is being worked on and what is left to do.
I hope this illustrates a good way to structure, manage and report on testing. The main advantage is that it is quick to do: it is extremely lightweight and can be easily added to and parts moved around with ease. It also doesn't need to be ‘read’. A multi-page document requires sitting down to read and it is left to the reader to try and interpret what you mean. A map can be glanced at and, almost immediately, the structure, connections, functions, what needs testing and what hasn't been thought of, are apparent. A map is worth a thousand words.