Sometimes we have multiple bugs that turn out to have the same root cause. And sometimes we know that they have the same root cause but we haven't yet fixed it. So the bug keeps happening. With a manual test, that's no big deal - we just don't log it again. But what do we do when it's an automated test and it just... keeps.... failing?
The Normal Case
If we're in a normal case, this is an isolated bug that only shows up in this same test. We just keep logging the failure into the same bug. This way, when we go back we can see how often it failed and if there were any other factors that affected the way in which it failed.
The Root Cause Case
It gets a bit more difficult when you have multiple tests failing. So now I've got two (or five or ten) bugs, and they have the same root cause. How do we handle this? We've got options:
- Close one bug as a duplicate of another
- Leave the bugs separate and just comment that they have the same root cause
- Merge or otherwise combine the bugs
- Create a bug for the root cause and link the different instances in (how you link is up to you)
We use the latter method and call it "creating a child bug". We log a new bug just for the root cause, and then mark all the other bugs as children. Then we keep logging errors into the child bugs.
The upside to this is that it's easy to see the root cause (and all analysis based on that) without cluttering it with test failures. The downside is that you have to go to each of the children to ensure that it really is fixed in all situations.
How do you handle this scenario?