Wednesday, November 9, 2011

Trusting the Code

I'm working on a project now that involves writing some automated system tests. These are pretty simple, in the grand scheme of things - functional smoke tests and small load tests. I write them, run them against QA until they pass (fixing the test and/or the system along the way), and then hand them off to the QA team for future use. Pretty straightforward.

And then.... building trust begins.

You see, after the tests have been handed them off to QA, they don't always work every time. Sometimes they fail because a new build broke the system. Sometimes they fail because we're pointing it at a different environment that is not configured correctly. And occasionally they fail because there's a bug in the test.

Regardless of why the test fails, when it fails, the first question is: "Is there a problem, or is the test just wrong?" This is where trusting the test enters the picture. It takes time and repeated usage to build trust in any code base - whether that's a system or a test.

So when something happens that's unexpected or undesirable - a test failure, a system response - don't be offended if the first thing you, the author, hears is, "Is it your code?". That's just trust that's not quite there yet. Figure out what's going on, and over time the test (or the system) will get better, and trust will come. Eventually, you'll stop getting asked, "Is it your code?" and start getting asked, "What went wrong?" - that's trust.

1 comment:

  1. Hi Catherine,

    I've recently been developing a system-level test suite for some significant new functionality (in a client-server system). As an experiment, I've been trying to share test data with Dev's unit tests. We agreed a common data format and now drive portions of both test suites (one Perl, one JUnit) from the common data.

    As you might imagine, we've had a few "Trusting the Data" discussions, with trust needing to flow both ways, but we're getting there and this experiment looks really promising.