Tuesday, July 7, 2009

Start Early

Upfront disclaimer: this is not strictly about testing. Sure, it applies to testing, but it's also about dev, testing, requirements elicitation, deployments, a lot of things. I'm going to use the word "release" throughout, but you can consider that to mean release or deployment or PRD or whatever applies in your situation.

Let's say we have a release. And we've decided that we need to do 10 things in that release. For simplicity's sake, we're going to call them things 1-10. Further, let's say you have a team with all the right stuff to do these 10 things. Congratulations.

Now, when we start the release, we have a decent idea of what things 1-10 are, and we have an estimate for each of them. Depending on how much you know, exactly, those estimates are what I think of as 30% estimates, or even 10% estimates. In other words, you're close (within +/- 30% or even +/- 10%, depending). But... you're not quite there. So you have 10 items, and each of them could be 30% short. That's a pretty significant schedule overrun. Maybe you were wrong the other way, but let's be pessimistic for a minute.

Basically, at the point when you start the release, you have schedule risk. The risk is based on all the things you don't know about each of the features - that third-party dependency you have that really isn't as easy as the documentation makes it look; the poor colleague who gets the stomach flu and is out for a week; the algorithm change that should have been simple but doesn't actually work so we have to rethink it. How do we increase risk? By knowing more. By developing. A feature is lowest risk after it's done.

Fortunately, we're all smart people and we know this. So we look at the perceived risk of each feature and we try to do the higher risk features first. And usually it works to a degree, but sometimes that "easy" "low risk" feature that you left until the end just blows you and your delivery date out of the water. Whoops.

So what can we do to mitigate this?

Start early.

There are several distinct points at which you discover that something is harder than you anticipated, and they cluster. They're right at the beginning, when you dig in and immediately figure out that this is hairier than you'd anticipated. They're about 2/3 of the way through, when you start to actually run this thing you've started to create and discover that it just isn't hanging together. And they're at the end, when you're working through the edge cases and discover that there is a major flaw you missed. What's interesting is that if you look at the frequency of those occurrences, the beginning and 2/3 through problems are much more common than the ones you find right at the end. (Note: Not scientific; this is just my own observation.)

So the trick is to get through as much of the total overall risk as quickly as possible. Which means you have to get about 2/3 through each feature as soon as possible. As you're planning your release, plan to reduce your risk as quickly as possible by doing your high risk things, and the high risk portions of all features as early as possible.

A note on risk points: risk at the beginning of implementation and risk at the very end are pretty universal in my experience. The third risk point is 2/3 through implementation for where I'm at, but you'll have to discover/confirm that point in your situation. You can find this by looking at the story and discovering where your burndown chart for that feature changed, or by looking at where the bugs start to show up, or by looking at where in the story the status goes from "making progress!" to "well, still working on it". In short, ask your project manager (or equivalent); he/she will have a good idea.

No comments:

Post a Comment