Monday, January 25, 2010

Ambiguous Specs

Sometimes I get a spec that is ambiguous. For example, the following is part of a spec I saw recently:

Users have widgets, and each widget must get a license from the central license server. To get a license, each widget must make an activate call to the central license server. ... When the widget license expires, the admin will extend the license, and the widget must then make a reactivate call to the central license server. ... If the user has licenses available for multiple widget types and tries to activate a widget, then the widget is marked ambiguous. The admin should assign the widget a specific type, then the user can reactivate that widget.

Forget for the moment that the work flow is a little odd. The word "reactivate" here is ambiguous. "Reactivate" can mean "use the reactivate call" or it could mean "call activate again".

So how do we disambiguate our requirement so we can confirm it works? We have options:
  • If we have access to the customer (or whoever wrote the spec), we can ask.
  • We can assume either works.
  • We can look at what was implemented.
Obviously, asking the customer is ideal. It's not always possible, however. Sometimes, even when you do get to the customer, the answer you get is, "I don't know" or "It really doesn't matter. Just pick one."

Assuming both cases are valid may happen to work. In the case of a public API that's trying to be really flexible, it may also be the customer's intent. I've certainly had customers that had the attitude that "if someone might assume it works like that, we have to support it." (This is very hard in practice, even though it sure sounds good!)

Or we can look for other sources of information, in this case, what was implemented. Looking at the actual code is a bit of a slippery slope, because it's dangerous to make the assumption that the developer both understood what the customer wanted and then implemented it successfully (or at least successfully enough you can glean the customer's desires from it). However, it's a good source of starter information. For small things, where neither behavior is truly wrong, often the customer doesn't care. Consistency is what's most important. And as long as the behavior the developer picked is consistent with the rest of the user experience, often the customer will be perfectly okay with that.

When faced with an ambiguous spec, remember that you have a source of information right there in the code you're facing. It's not always the correct answer, but then again, it's awfully hard to find a source of information that's right all the time. So take as a starting point what the code does, and validate it the way you would test any requirement. Evaluate it for consistency, efficacy, and reasonableness. If it's good, then it's likely that the code really is telling you what the application ought to do.

(Note that there still may be bugs. We're looking to resolve ambiguities here, not find total perfection just yet!)

1 comment:

  1. I think ambiguity review is a great way QA can add extra value to a development team. It can stop bugs from being introduced, it opens communication between developers, and integrates QA into the documentation process early on. Good post!