Tuesday, January 20, 2009

Chicken Little Meets Risk Analysis

No matter how hard we try, we ultimately are exposed to risk. We're writing software, and at least in some way this project/system/release is not like anything that's ever been done before. There are so many many variables - people, code, languages, feature and the understanding of features, holidays, snow or other inclement weather, sick days, heck, even building power! - that ultimately boil down to "what can we ship when and how well will it work?"

So what do we do?

The basics of risk analysis are something that have been written about extensively (there's even a whole society for this), and it comes down to:
  • identifying possible threats
  • determining the likelihood of each threat occurring
  • quantifying the impact of each threat
  • managing your risk - determine what can be done to minimize the likelihood and/or impact of each identified threat
This is great. But... ultimately it comes down to your gut. We'd all like to think that we go into risk assessment meetings that sound like this:
"Our employee unanticipated time off ratio is 10% for this group of resources. So for a team of 10 people, you can guess that 9 of them will be there and working each day."
"Okay, so our threat is employees not being present and working, our likelihood of that threat is approximately 100%, and our impact is 10%. So let's add 10% to our current estimates."

Instead, we often get into risk assessment meetings that sound more like this:
"Well, it's the flu season, so let's go ahead and add 10% or so to the schedule."

We're all just working from our guts. We can prepare by looking at similar situations, but ultimately I don't think any of us has invented a time machine or mastered the art of seeing the future. Some of us are better than others, and over time experience gives us an apparently uncanny ability to make reasonable guesses quickly.

So if we can't see the future, and we don't often have the good data we would like, how do we do risk analysis in a reasonable manner? Of course each project is different, but in general there are a few things that I've seen help:
  • Make this a group problem. Different people bring different perspectives and opinions. Oh, and make sure everyone is an active participant, not just a listener.
  • Don't be afraid to be a little bit Chicken Little.  In general there will be a lot of optimists in the room, so adding a dash of pessimism can provide some balance.
  • Don't be too Chicken Little. The point of risk analysis is risk mitigation - acknowledging the risk and then doing something to prevent or minimize it. If all you do is scare everyone, then you'll never get to the mitigation step.
  • Figure out what risks don't change and learn to handle those. Although each project has different risks, some (like our employee sick day example) are the same across multiple projects. For those kinds of risks, invest in measurement of actual occurrence so that you can refine your mitigation strategy over time. That previous sentence, by the way, was mostly a fancy way so say, "learn from your mistakes".
While acknowledging that risk is not something that can be fully contained, it's incumbent on us as software professionals - developers, testers, managers - to identify and mitigate risk. We have to at least try to meet our features and our dates, and reducing risks is one of the tools in our arsenal. 

1 comment:

  1. I found this article on the subject of risk to be worthy of consideration and reflection:

    http://www.edge.org/3rd_culture/taleb08/taleb08_index.html

    ---Michael B.

    ReplyDelete