Monday, June 2, 2008

Automated Automation

In the way of the blogging community, some posts on some random blogs create a lot of traffic. I ran across one of those today*, read it, and come to find out, there was an interesting nugget buried in there. The way I read the entry, Steve Rowe isn't just advocating test automation (which seems to be what he's catching a fair amount of flack for). He's advocating automated test case automation. I read this a lot like code generation tools, just for testing.

Whether to automate, when to automate, etc. is a rapidly dying horse the test community has been beating for a while (with great fervor on all sides). But code generation for tests? Now that's interesting.  

Code generation as a concept has been around for a long time; it's basically what compilers do. More recently, the concept has been used to generate more programmer-readable levels of code. Basically, a code generator will take elements and build them out into a more complete program using some sort of a model. The idea is that you provide the business specifics and it uses one or more mechanisms to create a consistent program that conforms to those business needs. There's a much better description in wikipedia.

So, how could this apply to tests?

Well, on a certain level there are some tests we write over and over again in a given project. In one of my recent projects there are at least 8 places where we perform what is essentially the same logic: if a member of a group uploads a picture for this journal entry, then use that. If they don't, use the group's picture. If the group picture does not exist, use the default picture. Launder, rinse, repeat, substituting "team" for group and "message" or "activity" for journal entry. Now, when I did this I wound up using a set of helper methods to actually do the work and check the rules. And it worked. But I'll have to do it again (and probably tweak the helper methods) when this project adds some other similar feature. Code generation could be an interesting tool here. It's such a small scale that I'm not sure it's worth it, though.

On another level, there are the tests that are done time and again in lots of different projects. Think login forms, for example. There are eleventy-zillion username/password/remember me forms out there. Wouldn't it be great to have a code generator that basically set up the tests for you? You just feed it rules about a valid username and a valid password, and it tries all the combinations for you. The combinations are well known (correct, incorrect password, nonexistent user, etc). And since the generator itself wouldn't contain any business logic, it'd be a natural for distribution to your local coding community.

I've never actually done this, but the notion - especially the second one, where it's useful across a fairly broad audience - might be worth a shot. It certainly doesn't solve all our testing problems, but I'm a big fan of getting the easy stuff out of the way, and if code generation can help us all do that together, then I'm up for trying it.


EDITED: I noticed the Visual Studio 2005 has some tools to generate unit tests. I've not used it, but it looks like it generates a positive test for each method. The tool I'm thinking of would be a much deeper test (actually test incorrect values and create multiple tests), but would only work in a given problem space.

No comments:

Post a Comment