Wednesday, March 26, 2008

The Weirdest Things Can Send You Sideways

There's an old cliche - "Expect the unexpected" - that should really define a good chunk of software engineering.

Everything's going great and then... your hard drive goes kaput.
Everything's going great and then... big snowstorm happens and the power goes out.
Everything's going great and then... you notice that another part of the system you happen to be testing just started behaving oddly.

For QA I'd like to tweak that a little and say "Embrace the unexpected". 

Finding the unexpected can short-circuit your schedule. It can change all your well-laid plans for the day. But it's also where you'll find your best bugs. One of the best ways to find good deep systemic bugs is to not look right at them.

So let's look at part of our arsenal of testing techniques*:
  • Basic Test Cases (Scripted). Useful for things like input validation. Generally these are things you're going to want to automate.
  • Exploratory Testing. Useful for discovering a system or a new feature of a system. Great for a complex system with a lot of different uses. Very highly dependent on having a good tester at the keyboard and takes a great deal of testing creativity. This isn't something for novices to try without additional guidance.
  • Session-Based Testing. Useful when you have a team of testers and need to manage it to a greater degree than exploratory testing. Test charters can provide some guidance as to what to test and (to a minimal extent) how. I've had good luck using this as an acceptance test with users (or other non-testers) involved. Difficult to set up correctly, but can be highly effective.
The beauty of embracing the unexpected is that it fits with each of these techniques, but it isn't a technique in itself. You're not following a script. You're not following a charter. You're not posing a question and getting an answer from the system. You're just... noticing.

So awesome. Use whatever technique(s) you like and are appropriate for the system. Just keep an eye out for the unexpected and learn to love the "that's weird" moments.

*Disclaimer: There are many many more testing techniques out there. In order to make this a blog entry and not a book I've limited this to the techniques I use most often.

No comments:

Post a Comment