Tuesday, February 23, 2010

Who's Going to Read Your Test Plan?

There are a lot of different ways to create a test plan. A test plan can be a simple list, or an excel spreadsheet, or some code, or a Word document conforming to your favorite test plan template, or the output of a test management system. All of them are legitimate, but some are more appropriate than others. You have to write a test plan, so what do you write? What's the proper format? The correct contents?

Guess what...? It depends.

There is a relatively simple heuristic for this:

The more remote your audience, the more information you need to provide in a test plan.

Let's say you're writing for your internal test team:
If you're writing a test plan for your test team, you don't have to put much information in, because a lot of it is assumed by the team already. You can skip the information about tools in use; the team is already using them. You can skip information about why you're using session-based testing, since you've probably already talked about that when you introduced it.

Let's say you're writing for a client who is consuming the software you produce:
The client is going to be looking for comfort that you know what you're doing and to make sure they cover any holes you didn't test. So you need to give them pretty deep background so they can see what you're covering and what you're not. You'll need to include some background on your test methodologies, what you choose to test, what you choose not to test, and why. The analysis you do to determine test breakdowns should be in the test plan as well. Don't be overly verbose, but pretend you have to explain all of this without talking to a person.

So, given those examples, things to consider putting in a test plan are:
  • Overview of your test methodology. Do you have a test philosophy? What is it? How do you fit in with development? What are your goals in testing? Use this when you're presenting to an audience that is not steeped in your methods (aka who doesn't work for you).
  • Analysis leading to test cases. Are you doing boundary value analysis? System modeling? Stochastic event modeling? How do you approach the analysis? What parts of the system do you use it on? Use this when you're presenting the test plan outside the test team, and when you're using a new form of analysis. Note that the analysis itself and the resulting tests may be documented separately.
  • Test ordering. Which tests will you do first? Which ones last? Why? This is important if you're at all worried about a crunch and possibly about not finishing your plan before you ship (in other words, do this one almost all the time!).
  • Timelines. Describe when you'll do tests and roughly how long each one will take. In particular, address overlap with other team work, like tests you'll do while dev is finishing another feature, or fixing bugs. This can be a reference to a schedule or other document.
  • Test categories. Describe the kinds of testing you'll do, including functional testing, performance testing, etc. These will be different depending on which heuristic you're using to break down your tests, but as long as you briefly describe each of the test types it's generally possible to translate between heuristics. Be sure to include tests you're explicitly not running and why. This one will almost always go into your test plan, since most audiences will need this (even your future self trying to remember the release!).
  • System model. It helps to provide a brief overview of the system from a tester's perspective. Perhaps it's an overview of the various interfaces, perhaps it's a set of use cases, perhaps it's a data flow diagram. This provides insight and a reference point for the test categories and analysis you do. Often the system model is associated with the analysis as it's performed, so it may be incorporated by reference.
There are some things that are not in the test plan, namely: test cases and test results. These are large enough items that I find them usually best contained in another (referenced) document and/or format. Put them in if it suits you, though.

The point of a test plan is to communicate information. It describes for some audience what you intend to do with the software prior to putting it into the field. That intention is driven by how you test software in generally and the particularities of the given release and/or system. Learning how much your audience knows will help you provide relevant information and share the reasoning an assumptions in your testing. In the end, that's what a test plan is about.

1 comment:

  1. Thanx Caterine for sharing test plan contents, your post helped me a lot.

    keep up the good work.