Wednesday, April 28, 2010

Testing Error Messages

There is a type of test in which the tester attempts to do something erroneous and makes sure that the software fails gracefully with a helpful error message. This could be anything from an invalid entry in a form field to a network connection error to an illegal operation.

The tests generally do something that should result in an error. Then they confirm that they get the expected error. There's one more step, though:

An error message is only complete if it provides the recipient with a course of action.

When an error occurs, the user hasn't been able to do whatever he was trying to do. To complete his task, the user must correct the error or find a way around it. The error message is the first guidance to doing this. The error message needs to be unambiguous and to correctly indicate how fatal it is.

For example, maybe a user enters the password "foo":
BAD: The password is invalid.
BETTER: Passwords must be at least 8 characters and contain one letter and one number.

Or maybe there was a network connection drop:
BAD: Error writing to server
BETTER: Connection lost to server foo. Please check your network settings and retry.

Or perhaps you just crashed a component of the system:
BAD: (Stack trace)
BETTER: A fatal error occurred in foo. Please contact support.

When you're testing (or writing, for that matter) error messages, make sure each error meets two criteria. It must properly throw an error, and it must be an error the user can correctly do something about.

1 comment:

  1. Another practice we like to use is to incorporate unique message IDs whenever possible (as opposed to using generic messages for common issues that may occur across multiple screens, databases, etc.) This greatly facilitates the trouble-shooting process.