Wednesday, January 27, 2010

Expect to Find

Creating test cases is really easy. I can do that all day long, and wind up with dozens or hundreds of 'em if I'm on a roll. For example, I'm currently thinking about how to tests a process that should run in a fairly isolated way on a node in our system. There are other background processes on the system, and then there are foreground processes that provide the core user functions while this goes on. So I can write test cases of the form "Run test process while background process Y occurs" and come up with a lot of them.

But is that really useful? In a truly black-box system where I don't understand the potential interactions between processes, I might have to run all of these. I'm not in a truly black-box environment, though; I understand the architecture of the system. Knowing the architecture of the system lets me intelligently pare down (or reorder) my test cases to the ones that are more likely to show any problems. I can find potential issues much more quickly than I could blindly poking at system functions.

Given a potential test case, to see if it's useful to run, consider the following:

What is an example of an issue that this test might find?

Simple, really. Just like I can imagine a lot of test cases, I can imagine a lot of bugs. Let's put that to use. The purpose of running this test is to gather information, often in the form of unexpected behavior. Speculating as to what information this test case might provide or what issues it might find will help me figure out if the test should be run early, late, or not at all.

For example, in my system when I have this test process running, I know that a process on another machine in the same system is unlikely to affect this test process, which runs completely on box with no external dependencies. I should confirm or refute that assumption, but that can be done with a few test cases; I don't need 20 test cases to prove the same thing.

So as I'm thinking of tests to run, I ask myself what this issue might show. That way I know whether I should run the test, or if it just duplicates something I've already done, or if it's just poorly designed. Speculate about what you might find; it'll save some time in the end.

1 comment: