Thursday, February 10, 2011

Definition Tests

When I get a new requirement, it's easy to read it and say, "sure, makes sense" and move on. Sometimes I later discover that I didn't really understand it at all. I just thought I did until I really sat down to think about it.

So to encourage myself to think about it, I do what I know as a "definition test" on the requirement. Basically, I go through the requirement twice times. The first time, I replace every key word with a synonym. The second time, I write a definition (like a dictionary style definition) for every key word in the requirement.

For example, I recently got this requirement:
Update documentation to explain variability in change rates due to probabilistic data ordering.

First, I identify the keywords: "documentation", "variability", "change rates", "probabilistic", "data ordering".

Replacing them all with synonyms, I get this:
Update the Integration Guide to explain inconsistency in "rate %" due to non-deterministic sequence of chunk ingest.

These synonyms are specific to my client. "Integration Guide" is a piece of documentation. Change rates are presented in our statistics, logs, and other outputs as "rate %", and data ordering refers to the specific sequence of chunks of data as it passes through our system. I show my synonym requirement to the product manager and to the head developer, and they think it matches. So far, I think I get it.

Then I take the requirement and write definitions for my keywords:
  • Documentation = one or more pieces of material intended for reading and use by customers and potential customers, who understand the high-level architecture of the system but who do not have access to implementation details.
  • Variability = inconsistent output given the same inputs, where inputs include configuration parameters, data set and order in which that data set is presented to the system, and hardware resources (RAM, CPU, disk) allocated to the system
  • Change Rates = a specific statistic in our system as represented in outputs as "rate %"
  • Data ordering = the sequence of chunks of data and specifically that sequence as it changes or does not change at various points entering and inside the system (e.g., a queue passing between two system components would be a point at which chunk sequence could be measured; a multi-threaded operation would be a logical point at which chunk sequence might be altered)
I take the definitions I wrote to the product manager and to the engineer implementing this requirement. If we still all agree that's correct, then we all agree on the requirement in much more detail.

If I still think I understand the requirement after I do a definition test on it, then I probably do. If product management, engineering, and I agree on all forms of the requirement, then we probably understand it in pretty much the same way. Try a dictionary test; it's a cheap way to make sure we all really do get it right.


  1. "Blogs We Like" sidebar: is there a problem?

  2. Is this a blog about software testing or technical documentation? You could copy this article, this sage advice, and paste it nearly verbatim into a blog about writing documentation it is so apropos.

  3. Anonymous: thanks - a couple of the links were stale, so I've fixed them.

    Steve: This is a blog about what I want to write about. I tend to write about testing, software development, management (especially technical management), all sorts of things. I've worked all over engineering organizations, and there's a whole lot of overlap in techniques and considerations. I'm glad it applies to technical writing, too!