Generally this look brings up other questions about what was going on in that release. Why did we skip performance testing that time? Why did we do extra tests around Active Directory integration? It's fairly easy to remember that for the last release or two, but it gets a lot harder the further back you go.
Our test plan should say why.
This would be a lot easier if we simply put the reasoning down in the test plan. There is an intent in a test plan, a set of circumstances that motivates you to look at a product in some ways and not in others. You'll forget that if you don't write it down.
Test plan motivations (or intents) can be very simple things:
- "Point release with tightly controlled changes, so only doing a smoke test in other areas."
- "Performance tests will not be performed because the new hardware to do it won't be in before release."
- "Many recent customer complaints about Active Directory ease of integration, so additional focus there."
Just write down why you're doing the test plan the way you are. In a year, when you don't remember the motives behind the plan, it will be very useful to be able to look it up.