Monday, June 28, 2010

An Error

What's wrong with this test code?

def testFailureMode
my result = runMyGreatTestCommandHere
if result.error
puts "Test PASSED. We wanted to make sure it errored properly"
else
puts "Test FAILED. Expected to error."
end
end

On the surface, it's not so bad. It runs a test that should produce an error, and makes sure an error is produced. Let's dig one step deeper.

We're testing a failure mode, which means we're expecting not just any error, but an error we induced. For example, let's say we're testing login with a bad password, and want to see that there's an error "incorrect password". However, this test would pass if any other error occurred. If the page never loaded, or returned a stack trace instead of a login page, this test would pass just fine. After all, it got an error!

When you're testing failure modes, make sure not only that you get an error, but that you get the error you meant to get.

2 comments:

  1. That's really a programmer rule too, not just a testing rule. Try { something } catch (Exception e) { more code } is so so wrong! There's a reason you can subclass Exception.

    ReplyDelete
  2. And make sure that the error makes sense, don't just make it check for the error that happens to be generated.

    ReplyDelete