Tuesday, October 5, 2010

What Will You Do With

Tests are good. I like tests. Tests show us things. Sometimes we call those things bugs. Sometimes we call those things information. Sometimes we call those things validation. Sometimes we call those things "someone should tell support because customers aren't going to understand this one".

The point of running a test is that we get something for our efforts. We have, after all, spent time, computing resources, brain cells, and possibly some money on this test. So whenever anyone wants to do a test, I ask one question:

What will you do with the results?

This question serves two purposes:
  1. It helps us understand whether doing the test is worth the cost.
  2. It helps us structure the test so we get the information we need from it.
In short, considering the output - including the reporting - before we start the test helps make sure its a good test.

For example, if I'm doing a performance test, I might ask "what will I do with the results?" If the answer is "give them to marketing for the website", then I'm going to design a performance test to give me the biggest number I can get out of a configuration that could exist in the real world (hey, it's marketing - we're trying to show off a little bit!). If the answer is "figure out what our biggest customer is likely to see" then I'm going to design the test to match the customer site and workload to the best of my ability.

Thinking about what you want - and it doesn't have to take long - will help make your test results much more usable. So before you start, ask "what will I do with the results?"

No comments:

Post a Comment