Friday, July 17, 2009

Customer's Oracle

In testing, we're attempting to state what a product does in various scenarios, and often to compare that to the behavior we expect in those scenarios. The differences generally turn into bugs, altered expectations, documentation, etc.

In a few cases there are specific requirements. Maybe you have a screenshot of what the product is supposed to look like, for example. Perhaps you have the exact algorithm that is supposed to be used to calculate the projectile's trajectory.

In many cases, however, we don't have an exact indication of what this thing is supposed to do. So we search out oracles. There are a lot of oracles out there, including:
  • Another piece of software you'd like to emulate ("do it like Windows does")
  • The sales guy or dev who's been around the block (and presumably knows what the customer does or wants)
  • A competitive product (even if you'd specifically like to not do what they do)
  • The spec for a standard you're claiming to implement
  • Etc...
Finding and using oracles is a supremely helpful thing. But.....

Is your oracle the same as your customer's oracle?

If you decide that you're going to do something "just like Windows", and your customer thinks the mac way of doing that thing is right, you're going to have some disappointment.

If you decide that you really really hate how many steps it takes to create a user in your competitor's product and you're going to do it in half that many, and your customer has trained some people to do that workflow efficiently, they're going to be really annoyed that they can't apply that training to your product. (This one is particularly true of products where admins and the like are highly used to keyboard layout and shortcuts.)

In short, you can (and should) pick an oracle, but make sure it's the same oracle your customer is using, or at least an oracle that agrees with your customer's oracle.

No comments:

Post a Comment