Friday, October 23, 2009


We work in what I imagine is a fairly typical environment. We code away on HEAD for a while. Once we're feature complete, we branch (so now we have HEAD and RELEASE). Then we fix stuff on HEAD and merge it to release until we hit code complete. We also go ahead with the next features on HEAD, but that's not currently the point.

The closer you get to code complete, and particularly after code complete, things get tricky. What do you merge?

There are, after all, several kinds of changes that might be candidates for merging into your branch:
  • Code changes to production code. Bug fixes, new features, etc.
  • Code changes to test code. Change the tests, not the code that actually ships.
  • Infrastructure changes. Change something underlying about the lab or environment (e.g., update the default fstab that gets installed)
So what do we do?

Code changes to production code tends to be the most commonly considered case. You evaluate the risk of the change, how much retesting you need to do, the benefit of the change (and how many of your customers are likely to benefit), and the amount of time left before you really really have to ship. Based on that, you choose to take it or not.

Code changes to test code are trickier. On the one hand, change is change and all change introduces risk. Sure, this code doesn't ship with your product, but it's still change. Plus, you have to consider risk here, too. If your test change breaks something, you might get less information out of your tests in the future, and that would be bad. On the other hand, it probably has some benefit, too: tests run faster so you can do more of them; or a test passes a failure point and lets you expose any other problems that occur later in the test; or perhaps you just spend less time looking at the error that isn't telling you anything new. For me, the bar to test code is lower than the bar to taking production code, simply because the risk to our actual (field) customers is lower, but there are some things that I try to consider:
  • Is the change caused by a failure or just a cleanup/enhancement/nice to have? The former is more likely to get put in than the latter.
  • Is the change going to fix something that causes problems for other tests? (e.g., a hang that stops all later tests in the suite from executing). A bad citizen like that is more likely to get fixed.
  • Is the change risky? The same types of analysis apply here as for product code. Avoid big, sweeping, likely-to-break-something changes.
  • How many more test runs are we going to have? The closer we get to the end, the longer we can just deal with the problem and not bother to fix it.
Infrastructure changes are generally not really optional. If you want your release tests to keep running in your infrastructure, they have to keep up with changes to your infrastructure. That being said, make sure you really need that infrastructure change, and be mindful of making the changes as small and safe as possible.

Merging is a tricky business, and the closer you get to a release the more of a "gut feel" kind of thing it turns into. So before you get into the thick of it, think about what you will merge and why. It'll save you some arguments later!

No comments:

Post a Comment