Friday, February 13, 2009

On Decision Making and Influencing

Michael Hunter (aka the somewhat-less-Braidy-than-before Tester) wrote a few days ago about the fallacy of the tester holding a release. His point, if I'm understanding correctly, is that somewhere our tester heads get filled with the illusion that we can go running down a corridor shouting "don't ship!" and the product will actually not ship. (He put it more eloquently than that; I'm paraphrasing.)

First and foremost, I agree with Michael if the tester we're talking about here is a member of the test team and not the manager or lead or whatever you call the most senior tester in the group. If you're a tester, you have a test manager for a reason, and foisting this kind of political hot potato off onto him/her is part of that reason. Take advantage of it. And if your test manager doesn't understand that shipping is not your decision to make, you have my sympathies and I hope the job hunt goes well.

However, if you're the test manager, does that illusion still hold? Do you still really get to believe you can decide whether to ship or not? And do you really still not get to make that decision?

I would argue that the problem still occurs. How many times have we heard the questions:
"Is that bug a blocker?"
"So, are we ready to ship?"
"When will you be done testing?"
"Is this build ready for [client]?"

There are a couple of things you can do here:
  • Leave. This has happened at every test job I've ever had, though, so I'm not sure leaving will fix it.
  • Refuse to play the game. Simply answer something like, "I don't know. I do not possess sufficient knowledge of the business circumstances to be able to make an effective decision." This stonewalling is not likely to make you friends.
  • Change the circumstances. Accept that you're going to be asked and spread your risk by making your voice just one among several contributing to (but not making) the ship/no ship decision.
I generally vote for changing the circumstances. Which is nice, but how?

Recognize why your knowledge looks special
It helps to understand why you're being asked these questions. At this point in the product life cycle you the test group are the ones with the most knowledge about the actual product. Product management can talk about what it ought to do, development can talk about what they tried to do, support can think about how to interact with it, but test is the group that can say what it actually does (at least in some cases). You have first hand knowledge about the product and that gives you a loud voice. It would be stupid of the major stakeholders to not at least seek the opinion of the group that has more experience than anyone else with the product, even if that experience is not yet sufficient to make an informed decision.

Make the decision communal
This is the single biggest thing you can do to alleviate the "testers pick what ships" fallacy. You can call it a release board, or a product team, or a release management committee. This should include all major release stakeholders: sales, marketing, product management, support, and development (with QA). And this group is what says yes or no on a release (or in some cases, makes a single ship/no ship recommendation that upper management then signs off on).

Getting this started is always fun, and you'll need your boss's help to make it more formal, but it's eminently doable generally because it sounds reasonable. Plus, getting all the stakeholders in a room to make a decision lets you say I don't know. After all, you're not saying, "I don't know." You're saying, "I don't know but here's how we can find out," and that's a much more actionable response.

Discuss likelihood and workaround
When you go to talk about untested areas in a release, lack of knowledge is not a very compelling argument. Instead, talk about likelihood of there being customer-affecting problems in untested areas, and what kinds of workarounds can be done. For example, if we haven't tested writing using WebDAV, can we ship the product to our CIFS-only customers first and ship a bit later to those using WebDAV?

Similarly, when you talk about a bug, all anyone really cares about is how likely it is to happen, how the customer will be affected (are we talking total meltdown or a small error message in a remote area of the product?), and what can be done to avoid or fix it. So talk in those terms. Saying, "We will see this at [customer], they will lose data, and we don't know of a way to recover it" makes your issue really likely to get voted a stop-ship problem.

Educate yourself
Remember, you're one of the loudest voices in the room even if the decision isn't yours. Your analysis needs to be educated and fact-based. Talk with sales and with marketing to minimize surprises. When the schedule is first created, ask why those dates? Is this feature based, or is there a major trade show or some revenue we really want in that quarter relying on shipping this software? Criteria for shipping can definitely change based on what's waiting for the shipped product and how urgently. This will also allow you to identify reasonable workarounds and be prepared for questions and concerns.

Don't involve the whole test team
In general it's not good to have each tester (or each developer for that matter) running around talking about whether a release is shippable. This is what you have a test lead or a test manager for, so use that. Testers should make their case to their manager, and let the manager speak for the collective team. This will minimize the political playing off each other that mostly takes a lot of time and gets teams upset.

Rely on your reputation
Being asked to make or strongly influence a decision is not a good position to be in, despite the frequency with which you the test manager will end up there. You need to have a good strong reputation so when you talk about risks you get listened to. If you've been right quite often (regardless of whether the ultimate decision went the way you wanted it to), then you will be listened to, and when you say you need more time to reduce risk you're more likely to get it.

Work offline
Don't save these kinds of decision requests for meetings. In a public forum like that it's easy to back someone into a corner and being reasonable flies right out of the window. A quiet word of warning about the state of the release can turn a public disagreement into a private convincing - and the private convincing is much more likely to be effective.

1 comment:

  1. I hadn't thought about it too much, but I went in hoping for a strong advocacy that "QA" -- if not a tester -- should decide if a product is shippable, and lamenting that it never really is so (we just get blamed if it ships with bugs, documented or not.)

    But now I agree with you. It's a business decision. QA should be a principal informer, and we should be arguing about any reservations we have, and not feel like we're "holding up" a release.

    I've been in too many late evening sessions where management pressure is waiting for my "blessing." And there is a lot of pressure to justify.

    Our job is to present the negatives, and let someone with the proper perspective (and responsibility) make the decision. But of course, if I'm given the responsibility (and commeasurate reward), I'll be glad to make the decision. I might even put on a suit for the shareholder meeting -- I hear you don't have to wear a tie anymore.