Monday, November 14, 2011

Your Test Code Should be Defensive

One of the things you learn when you first start to write code that will be used by others or with other code is this: "write defensive code." It's shorthand for not trusting inputs or external dependencies, but checking them before you try to use them, and it involves things like validating inputs, calling external services asynchronously, etc. Well, for all you test automation engineers out there:

Test code should be very defensive.

After all, we're running test code because we want to make sure that an external system (the system under test) does what we expect. If we knew beyond the shadow of a doubt that it worked, then we wouldn't bother running the test. So, we're only running the test because we don't know with 100% confidence that it will work. And that's when we have to be defensive in our coding. We're already looking for things "that should never happen", so let's write our test programs so they can handle them.

Make sure you at least consider whether you  might need:

  • Asynchrony and/or timeouts. Does the system not returning mean your test will hang?
  • Null checks. "That should never be null" is a lot easier to trace if you check when you first see it rather than waiting for it to blow up somewhere later in your code.
  • Try/catches. "The program should never throw an exception", key word "should"
  • Results inspection. Just because the program returned something and didn't error doesn't mean that return matches what you thought.
Basically, your goal is to either produce a "pass" or a legible failure. Keep in mind that automated tests ted to (1) have a long life; and (2) be run mostly by people or systems other than the ones who wrote them. Just because you the test author knows what it means, that doesn't mean it'll be obvious to the QA engineer evaluating the output of the results a year from now (even if that QA engineer IS the test author!).

Keep your tests defensive, keep your error messages useful, and let your tests have a good long life.

1 comment:

  1. I wish there were more automated test design tips like these generally available.

    One I like a lot that applies to test code only is "don't use conditionals". If you don't know for sure which branch of an if() statement the test is following, one of the branches could erroneously disappear completely and the test would never show it.