Friday, January 4, 2008

Who Cares About Test Types?

I've been thinking about different kinds of tests and their audiences. What do these kinds of tests tell us, and to whom is each speaking?

Unit Tests
First, a working definition: unit tests exercise internals of the code, typically on a per-method level. These are also called white box tests.

These tests are useful for:
  • Identifying that your method does what it thinks it does. This is particularly useful when you are doing math or logic inside your method. It's not so useful when all your method does is pass along an input to an output.*
  • Confirming that your method takes what it needs to take an outputs the correct things. The act of writing a unit test will force you to think about your function, and that can catch out the silly mistakes like needing an input you don't have, etc.
  • Testing basic code practices. Unit tests are a great place to check that you handle nulls, empty strings, etc. properly.
System Tests
Again, a working definition: system tests use the system as your customers will, whether your customer is another team within the company or external, and whether your interface is an API, a GUI, a CLI, or whatever. These, ultimately, are the tests your customers care about.

  • Confirming that the system meets a customer's needs. These tests prove that when the customer does something, the proper thing comes out the other end. How you got there doesn't really matter; that you got there (in a reasonable time and with reasonable resource usage!) is what this test cares about.
  • Confirming that you can really handle external users who don't know the subtleties of implementation. Since users don't know the internals of your code, they'll send in odd values, missing values, null values, etc. These tests confirm that the system can handle that.
In the end, your customer only cares about system tests. They're what tells your customer whether the system will do what it's supposed to do. Unit tests tell you about how the system does what it does. These are tests that are for the developer, and provide a good cushion under which the system can grow and change.

* I know some people disagree with me on this, and it's certainly simpler from a code coverage perspective to unit test all methods. I don't think it's harmful, but it doesn't seem overly helpful to have a unit test that simply reimplements the method. Net net, if you have the time or if you do a lot of refactoring, then go for it. If you don't, then just be sure you test your high-risk methods first (just like all testing, riskiest first!).

No comments:

Post a Comment