Situation-based scenarios are those where you create a situation and see what happens. This is often what we think of when we say we're testing something. We decide to log in with spaces in the name and see what the system does (appropriate error message, let us in, blow up dramatically, etc). We create a degraded hardware scenario (translation: yank out a hard drive) and see how long it takes the system to send a notification while we induce heavy load. All of these are scenarios where we started with the situation, and we are looking to see what might happen.
To be effective in a situation-based scenario, a tester needs certain skills:
- Creativity in imagining things that might happen (coming up with scenarios)
- Ability to analyze a system to identify interesting actions and potential interactions
- Ability to closely observe a system and identify resultant behaviors, even when they're not directly visible
Conundrum-based scenarios are those where something happened, and now we get to figure out what on earth went on. When you get a problem in from the field, or a bug reported to you, you're looking at a conundrum-based scenario. You have a user who was "just writing, like normal", and the system crashed. Or you have an error message in your logs that development assures you "shouldn't happen", and you don't know how to get there. In conundrum-based scenarios like this, you are given the result, and you need to describe the situation.
To be effective in a conundrum-based scenario, a tester needs a different set of skills:
- Ability to analyze a single state and identify relevant facts (e.g., the core file, not the log file name)
- Grep and other analysis skills
- A mental model of the system to identify operations and other actions
- Creativity in scenario imagination
Working with situations and working with conundrums requires complementary but slightly different skills. When working on a problem, figure out which one you're facing. You'll be far more effective if you know what skills to apply.
2 comments: