Monday, April 7, 2008

Code Coverage Complexity

I'm working on a project that is looking at using rcov for code coverage. Makes sense; it's a Ruby on Rails project, so rcov is easy to integrate and even ties nicely into the build system.
So what kind of code coverage do we want? Easy! 100%! Every line of code must be tested!
Err.... let's talk about that one for a minute.

Different Levels of Coverage
There are several different types of code coverage: C0, C1, and C2.
C0 code coverage tells you whether a given line of code was executed.
C1 coverage tells you whether each branch of each line of code was executed.
C2 coverage tells you whether each code path (across lines) was executed.
For example:
foo ? bar : baz
Let's say I have just one test (pretending for the moment that this is at all complete):
def test_foo
    foo = true
What's my coverage?
C0: 100% - I executed that line
C1: 50% - I executed only one branch of that line - the true path.
C2: unknown, but no higher than 50% (it depends on the rest of the program, which isn't shown here)
Different Types of Code
Coverage tools include and don't include various types of code. Depending on the tool you use, it might include third-party libraries, or not include view code, or not include javascript.
Coverage Goals
So with all of that, what's a reasonable coverage goal?

Well, that will ultimately depend on the tools you pick. So, in code that you own, and in code that you can measure, aim for C0 coverage of 90% or better. Go for it. The more complex the coverage (C1 or C2), the lower the coverage level you can expect.

1 comment: