Wednesday, October 31, 2007

Which Came First?

My current job uses an XP (aka Extreme Programming) development process. Now, lots of companies say they do, but these guys really do, right down to stories, a force-ranked queue, pairing, and writing tests first.

There are a lot of tenets of XP that affect test, but the two that I would like to consider today are:
  • Test first.
  • Automated tests are good (yes, this is a simplification, but parsing this better is a job for a later post)
This is all well and good, but what should you really be testing first? There are three ways to approach testing a feature:
  1. write the automated tests, then test manually for usability and context
  2. test manually, then write automated tests for regressions
  3. split automation from manual testing and try to accomplish them roughly in parallel
Extreme Programming lends itself very well to approach #1. You write the test automation as (or even before) you write the code. Then, when it's basically done, you take a look at it as a user would see it - through the GUI (however that's expressed).

More traditional software development methodologies put test automation on QA's shoulders, and approaches 2 and 3 are very common. You look at a feature first as the user would see it, then you automate.

So, now that I find myself in an Extreme Programming shop, how do I change the more traditional approach? Or how do I change the XP approach to provide some of the benefits of the old way?
  1. Do not eliminate manual testing. Yes, I know this violates XP methodologies, but if you don't see what your user sees at some point, then you will make a product that is very hard to use. If you can figure out how to automate the user experience once you're happy with it, great, but do have the courtesy to step into your user's shoes at some point.
  2. Do allow automation to reduce your manual testing time. Automation can't cover everything, but it can eliminate many classes of tests. No more manual boundary value testing of that form - just automate it and move on. Save the manual tests for the things machines can't do.
  3. 100% defect automation is good. If you can describe it well enough to be a defect, you can automate the test for it. This saves a lot of time in regression testing in particular. I wrote a whole blog post on this earlier.
  4. Automate the boring stuff. All those tests that have to be done for every release or on every build and that are the same over and over are good candidates for automation. Humans make mistakes and take shortcuts because they believe they know what it will do, so make the computer do it instead. This also frees up your humans for more advanced, more interesting, more complete testing - and keeps your team engaged.
  5. Test as the user. Figure out what tests of yours users really care about and hit them manually, at least sometimes.
Put these all together and you've accomplished two pretty important things. First, you still put your user at the center of your testing. Second, you build a base of test automation so you can test more and test better.

No comments:

Post a Comment