Friday, September 4, 2009

Customer Demos

One of the things QA handles where I work is demos and evaluations of early stage features. For example, a potential customer might need a feature and come in to see an early draft of it, or might take a beta system in house to make sure the feature does what he expects. Generally there's a fair amount of pressure on these evaluations because a sale or analyst presentation or something similar riding on it: "If it goes well, we'll buy/present/rave about you."

Okay, take a deep breath. High pressure. You can deal with it. There's only one rule:

Don't do anything you haven't done before.

That's it!

Basically, an evaluation is a big demo. So use your internal demos to prepare for it. You won't be able to control everything (citywide power outage, for example), but you can control a lot of things (which flags to use when starting the product, for example). By trying everything on your own first, you can make sure the things in your control go right.

When we test a product, we try to avoid ruts where we're just doing the same thing over and over again. With a demo, we want to find a rut. We're looking for a well-worn path here, and we're doing it by making sure we've tried everything before the demo.

So my evaluation preparation goes something like this:
  1. Get an idea of what needs to be shown
  2. Create a list of all the things that we'll show (down to test cases)
  3. Add the exact commands I'll be running and data I'll be using to that list. In the demo I'm not going to type; I'm pretty much just going to copy and paste.
  4. Run those commands on the build I will be using in the environment I'll be using (or the closest I can get to that environment). Time these.
  5. Repeat step 4 until I feel like I can do this in my sleep.
It takes a lot of time, but by practicing first, demos and evaluations can be approached with confidence and with a much lower rate of screw ups. It alleviates some of the pressure, at least for me.


  1. Hi catherine,

    Well Rehersed demo presentation always brings positive result, you are spot right right we should avoid things that we have not done before,to avoid last panic hours

    Keep it up :)

  2. This is why customers should not believe demos, and also why customers should never let a vendor do their own "customer acceptance testing."

  3. Ouch! I didn't think of it that way at all, James. You make practice and trying things beforehand sound like a bad thing.

    I simply think of it as one more way to make sure you're not surprised, or doing something dumb in front of a customer (or analyst, or potential customer). After all, just because a baseball hitter practices his swing before a game doesn't make the home run any less real! It just means that he practiced beforehand.