Tuesday, April 27, 2010

Reopen vs Duplicate

Sometimes you'll find a bug that looks a lot like an existing bug. Maybe it's the same test. Maybe it shows the same symptom, even. For example, maybe you have a test that starts, connects to a remote server, gets some information, and checks that information. The test is failing with a "connection refused" error, and that sure looks like the "connection refused" error you saw last week.

At this point, you have two choices:
  • Log it separately, and risk creating a duplicate
  • Log it in the existing issue, and risk conflating two issues into one
Either way, you're taking a risk: of a duplicate, or of a false reopen. So what do you do?

Go with the risk of a duplicate, because duplicates are cheaper.

Let's say that risk is realized, and the bug is a duplicate. Now you have to close the bug as a duplicate, and reference the duplicate bug from the original. Annoying, sure, and a little bit of time wasted confirming that the bug is a duplicate, but in the grand scheme of things, not a big deal.

If you'd gone the other way, and reopened a bug, you might have realized the risk associated with that: maybe it's a different bug. In this case you have to open a new bug, comment/edit/delete the erroneous reporting in the old bug, close the old bug again, and move any referenced logs or other information. Oh yeah, and now the developer who closed the bug is just a little bit insulted that you thought his fix was incomplete.

If you're very sure that the same root cause underlies two failures, please go ahead and reopen the bug. If you're not sure - like our "connection refused" example, which could have several different causes ranging from network failure to configuration problem to server failure - go ahead and log a new bug. It's easier to find out you have a duplicate than to peel apart two bugs that got conflated into one.

1 comment:

  1. Jerry Weinberg offers another way to think about. In QSM Vol. 2 he cites Musa, Iannino, Okumoto from Software Reliability: Measurement, Prediction, Application:
    A failure is the departure of the external results of program operation from requirements.

    A fault is the defect in the program that, when executed under particular conditions, causes a failure.

    Distinguishing faults from failures thereby establishes a clear language. From my perception, I have most often found failures in the products I tested. Since I did not know where the problem occurred, I couldn't tell whether or not I had to deal with the same fault. Oh, and it should be obvious once you grasped the separation, that multiple failures may have the same fault.

    So, finally, thinking about bugs filed by testers as failures, supports your statement to open another bug. As long as you don't know whether it's the same fault, you might confuse the developer fixing the bug.

    ReplyDelete