Transient Metrics

Metrics is a dirty word in some circles. That word and the underlying idea has come to represent turning off the brain, ignoring context, and blindly following some statistics that don't really tell you what you thought it did.

Now, don't get me wrong. Bad metrics can do a lot of damage. Most metrics are best used as guidelines rather than strict rules.

But.

What are we really looking for with metrics? For the most part, we use metrics to better understand something - a trend, a state, a readiness. Metrics can be an estimation and risk analysis tool.

The trick is to find a metric that answers the question you're trying to ask.

Often, this is a good way to approach a problem, where someone is asking you a question or asking you for a metric. Figure out what the question is, then derive the metric that shows it. It helps to say, "this metric is transient; we will not track it after X". Make your metric about data mining - gathering information to answer a question - not about rules.

The questions we ask vary over time, so the metrics we use to track them should vary over time. It's okay to use a metric for one release or even one day, and then throw it away. Gather the information, use it, and then throw it away. Don't be afraid of metrics, and don't be a slave to them. They're tools. Use them for as long as they're useful.