Monday, March 8, 2010

Estimates, Not Ifs

"How long will this test take?"

It's a reasonable question. The answer is generally reasonable, too.

"It'll take about 3 hours to set it up, then the test itself should take a few more hours. So, it'll take about a day."

There is a silent "if" at the end of that answer, though. After all, a lot could extend the time:
"... if the I can get on the hardware I need, which is currently being used by another test."
"... if the bug that blocked this test is really fixed."
"... if I don't get pulled off to another meeting between now and then."

As testers, we get used to providing information, so it's tempting to outline the ifs as well as the estimate. Most of the time, you shouldn't.

A test estimate is just an estimate. As the estimator, it's your responsibility to account for non-optimistic scenarios: all those ifs. So for each if, you get to consider how likely a problem is to occur, what you'd do if it occurred, and what that means for how long it will take you to do the test. Maybe all of those things are extremely unlikely; maybe something blows up 80% of the time (ouch!).

In either case, your estimate should now look something like: "About two days, allowing for the usual hubbub of life, plus the few hour setup time, and leaving time to run the test itself."

When you're providing an estimate, keep in mind that you need to describe the estimate, not all the things you can think of that would go wrong. Account, don't over-elaborate.

4 comments:

  1. "A test estimate is just an estimate."

    I agree. Unfortunately, the people asking the question don't always agree!

    To some people, it means "tell me when I can ship". To others it means "give me a committment for when everything will be done". For still others it means "tell me when you will be available to work on a different task".

    As always, understanding the context will guide you in answering most effectively.

    ReplyDelete
  2. Dang blogger ate the comment again. From Joe:

    "A test estimate is just an estimate."

    I agree. Unfortunately, the people asking the question don't always agree!

    To some people, it means "tell me when I can ship". To others it means "give me a committment for when everything will be done". For still others it means "tell me when you will be available to work on a different task".

    As always, understanding the context will guide you in answering most effectively.


    (From Catherine):
    Joe, you're absolutely right. Understanding the real question rather than the asked question will color your answer, or even tell you if you have the information to answer it.

    ReplyDelete
  3. "For techies estimate is a guess, for business is a commitment..."

    ReplyDelete
  4. For some theory, Steve McConnell distinguishes between:

    estimates (tentative or preliminary calculation), targets (desirable business objective), and commitment (promise to deliver defined functionality at a specific level of quality by a certain date).

    I think that's rather ideal, but it's still useful to keep in my what different things you are talking about when answering "when will it be done/do you have an estimate".

    ReplyDelete