Wednesday, March 30, 2011

Confidence Indicators

Talking is such an incredibly valuable thing. Whether it's in person, by phone, over IM, or even email, having a conversation can tell you so much more than looking at a piece of documentation ever will. As a tester, if I can get a quick overview of the feature from the development team, I can quickly learn how buggy that feature is going to be - just from how they talk about it. As a developer, when the product manager explains a feature to me, you can hear in his words whether it's well thought through or if I should plan some extra time for rethinking and reworking and integration. I think most of us get those kinds of senses over time.

But how do we know?


Some phrases and actions indicate confidence, and other indicate a lack of confidence. So when a product manager talks about a feature with confidence, he's probably thought through it. When a developer talks about a feature with a lack of confidence, then there's a hole there: edge cases, poor integration, something.

Indicators vary by person, and take some time to learn for each person. However, there are some common ones:

Confidence Indicators:
  • Direct, relatively short answers
  • "Sure" or "No problem" in answer to a question
Lack of confidence indicators:
  • Hand waving (yes, actual literal hand waving!)
  • "Well,..."
  • Repetition of previous results
  • Asserting something works without describing how it works
Sometimes it seems like there just isn't time to stop and have a conversation at a transition point. But take the 10 minutes and have the conversation. It's a rich source of information..... as long as you use it.

Tuesday, March 29, 2011

Transient Metrics

Metrics is a dirty word in some circles. That word and the underlying idea has come to represent turning off the brain, ignoring context, and blindly following some statistics that don't really tell you what you thought it did.

Now, don't get me wrong. Bad metrics can do a lot of damage. Most metrics are best used as guidelines rather than strict rules.


What are we really looking for with metrics? For the most part, we use metrics to better understand something - a trend, a state, a readiness. Metrics can be an estimation and risk analysis tool.

The trick is to find a metric that answers the question you're trying to ask.

Often, this is a good way to approach a problem, where someone is asking you a question or asking you for a metric. Figure out what the question is, then derive the metric that shows it. It helps to say, "this metric is transient; we will not track it after X". Make your metric about data mining - gathering information to answer a question - not about rules.

The questions we ask vary over time, so the metrics we use to track them should vary over time. It's okay to use a metric for one release or even one day, and then throw it away. Gather the information, use it, and then throw it away. Don't be afraid of metrics, and don't be a slave to them. They're tools. Use them for as long as they're useful.

Monday, March 28, 2011

The "We're Still Gonna" Method

One key thing to remember about most changes is this: someone liked the old way, and got value out of the old way. So when you want to get rid of the old way and put in the shiny new way, well, that value to that person goes away. Ouch!

How do we sell a change to someone who is losing value?

Simple. Make the change. Preserve the value.

I call this the "we're still gonna" method of change.

The idea is that we define the value in the old way and preserve it explicitly. We say, "we're still gonna have value X. We're gonna get at it by technique Y."

For example, we switched from CruiseControl to Jenkins for a build system:
  • Threatened Value: chat notifications of failed builds. Solution: Jenkins knows how to chat. We'll set it up.
  • Threatened Value: builds three branches (including two older ones we've shipped and still have to support). Solution: Leave cruisecontrol up for older branches.
  • Threatened Value: error logs available on main build page for easy access. Solution: link to error in build log on main page.
The idea is to ensure that we preserve the value even if the method of getting that value changes a bit. This helps make the change more palatable, and it also helps us make sure we're not giving up more than we want to just so that we can have something new and shiny.

Friday, March 25, 2011

Rules Are Comfortable

It's amazing how many rules we make for ourselves. We have rules of thumb; we have best practices; we have laws and regulations; we have habits - and they're all rules of varying stringency.

We also break rules. Frequently. I have a habit (a rule) of going to the gym every morning before work. This morning, I slept in and broke my habit. No gym for me this morning! Now, this was just a habit, so breaking it wasn't a huge deal. There aren't a lot of consequences for that breakage.

In 1995, Montana got rid of numeric speed limits. Going 100 mph wasn't necessarily speeding, as long as it was "reasonable and proper." And people hated it. They made statements like, "two years on the road with the crazies out there" made them afraid they would be "going along at 80 and someone from out of state, who doesn't understand the road conditions, passes you at 100." (source). Eventually, Montana imposed numeric speed limits again. They had their rules back. The only flaw in Montana's reasoning is that with no speed limits, there were actually fewer fatalities. Having no speed limits was safer, but having the rules made people feel better in the abstract. Oh, and people still speed; they just like to have speed limits around.

So how does this apply to software?

Well, instead of "speed limit", let's say "rules to release software". This gets pretty interesting. Many (most?) companies have rules about when to release software. These rules (or guidelines, or best practices, or targets) tend to take the following form:

Any software we release can have 0 P1 defects, and no more than X P2 defects. Further, the software must have successfully passed Y tests for 1 week prior to release.

There are of course variations on this theme. One commonality, though, is that the rules are bent or outright broken as the release date approaches. That defect that was a P1 last month but still hasn't been fixed.... well, that really should be a P2. The tests for a week prior to release..... well, maybe 3 days is fine.

We make the rules. Then we break them. But we still have the rules we make.

Why? Because it makes us comfortable.

With a rule for releasing software, or a known end point, we know we have to consider the release criteria: the number and priority of open defects, the state and stability of testing, etc. We may choose to change our criteria at the last minute, but at least we know those criteria are there and therefore we keep an eye on them. Rules for releasing software are guidelines to where we look and to things we consider. In the end, a release decision is always a gut decision. Someone holds her breath and says, "yes, let's release." Having rules and criteria helps guide that gut decision.

So don't throw away your rules. And don't expect to stop bending or breaking them. Just understand what the rules are giving you: comfort and a framework for making decisions. No more, no less.

Wednesday, March 23, 2011

Acceptance Leading to Change

Shocking news: most people would rather feel validated than attacked. They want to feel good about the work they've done and the ideas they have. That's part of what is scary about a new person or a new position - it is an opportunity for embarrassment, chastising, and being told you're wrong to occur. And that's uncomfortable.

I talked with a woman last week who is the office manager for the president of a local college. Her boss - the president of the college - is new, and she came to the school with him. There's an entire office staff that was already there. They already had procedures, and ideas, and methods. The new office manager and the new president also had ideas.... And things were gonna change. This is a big scary moment.

This kind of moment happens whenever there is a change. When you're the new boss, or when you make a process or technical change. If you're the new manager or the new change agent (consultant, coach, engineer, whatever), it's up to you to make sure that several things happen:
- the needed change occurs
- you still have a team when you're done (no one quits!)
- that team can still work together effectively (no one hates each other!)

The key is to start with acceptance. The old way was not 100% bad. There is always something good in there. Start with that. Accept the good in what the team was doing, and use that to keep people engaged throughout any change. If you can avoid "no" and "wrong", you're doing well. Instead, use "yes" and "like but with a twist" statements to make change palatable. Leading questions are also useful to help the problem identification and the idea come from the team itself, not from outside. The team will embrace and fight for a change they want, and they'll band together to do it. That way you get both the change you need and a stronger, happier team.

It's not enough to make a change. You have to make the change in a team-preserving way. Accept the team, make sure they feel safe to go through change, and you'll come out much stronger in the end.

Tuesday, March 22, 2011


Today at STPCon someone said, "He did a very good writeup. He is a senior tester." And that really made me think. As managers, we often say that someone is a senior tester, or that we're looking to hire a mid-level tester. This is a proxy term; when we say "senior" or "junior", we really mean that person has a certain cluster of characteristics.

And every person is exactly that: a cluster of characteristics. They are really good at some things and less good at other things. When we assign a seniority rank to a person we are really describing which characteristics we value. We - consciously or unconsciously - assign value to various characteristics, then determine which of those characteristics a person has, and only then open our mouths to say, "this person is senior."

"Senior" is not a gift or a word of praise. "Senior" is an expression of multiple underlying values. Embrace the nuance and explore the characteristics that make up "senior".

Wednesday, March 16, 2011

It Used to Be Worse

Sometimes software projects are great and wonderful. You start from the proverbial blank sheet of paper, and you "do it right". You understand your features, and you write your unit tests, and you set up continuous integration from the beginning. You even write your documentation as you go along! The whole team is happy, productive, and proud of their work.

Other projects aren't as nice.

You start with inherited code that you only sort of understand. Most of what you know is that its resemblance to a plate of spaghetti is remarkable. There aren't really any requirements or stories, the documentation and tests are sketchy at best, and this project just sucks.

You start making it better, one thing at a time. A little refactor here, some documentation there, a new continuous integration server that actually builds the monstrosity... errr.... project.

A few months down the line, some things are better. The project is starting to shape up a little bit. You're not out of the woods yet, though.

This is the point at which morale starts to decline.

You've been working on this nasty project for what feels like ages, and it still sucks! There are still classes there that no one understands. The test coverage is still spotty at best. The documentation represents a best guess, but no one really thinks it's accurate. And every week there's still some new discovery that customer X really needs an extension to feature Y.... and you didn't even know there was a feature Y.

If you were two years old, you'd have a temper tantrum right about now.

It's up to us to keep ourselves out of the doldrums and to get morale back up so we can keep going.

And here's the speech:
"Yes, this project sucks. You know what? It used to be worse."

That's pretty much the whole thing. Back it up with "remember how it used to.....?" comments - as many as you can.

The key to getting out of this kind of a morale dip is refocusing not on how bad things are now, but on how far you've come. Take a few minutes and remind yourself of the "bad old days". It'll make the present look a whole lot better.

Monday, March 14, 2011

Responsibility of the Teacher and the Student

I sat down a few weeks ago working with someone on a piece of software that I knew fairly well. He was totally new to the software and to the problem space. So, I sat down to "teach him the software", and he was on the other end of a Skype call learning the software.

This happens every day all over the world. Sometimes it's a success. And sometimes it's a complete disaster. Heck, even in just the teaching and learning I've been involved in, there have been great successes.... and really frustrating sessions. Over time, I've learned that an interaction between a teacher and a student is effective if both parties meet their responsibilities.

The responsibilities of the teacher:
  • Know your subject. If you're shaky on the topic and have to frequently say, "no, actually, hang on...", then the lesson will not go well.
  • Structure the lesson. Spend some time before you start thinking about the order in which you will proceed. Otherwise, you'll find yourself jumping back and forth, and that's frequently confusing.
  • Open a feedback loop. Don't assume silence indicates understanding. Stop frequently and ask if there are questions, then wait for a response.
  • Ask for understanding statements. Stop periodically and provide impromptu questions or exercises for the learner. It will help both you and the learner make sure that understanding is really there.
  • Do it on the learner's computer. If you're learning something like a tool or a language or a set of source code, working on the learner's computer ensures their computer is set up correctly to apply the lessons they're learning.

The responsibilities of the student:
  • Ask questions. The minute you don't understand, stop the teacher and ask. The teacher won't know you're confused if you don't say something.
  • Focus. It's easy to keep IM open in the background or check your email and figure you'll pick up the thread later, but you'll be much more likely to be confused. So close the email and chat, and focus on the lesson.
  • Practice. Once the lesson is over, apply it as quickly as possible. Practice solidifies the things you've learned. Plus, it's a lot easier to go back with questions right after the lesson than weeks later.
In any teaching situation, everyone involved - both the teacher and the student - has responsibilities. Accept your responsibilities, and the lesson will go much better, no matter which side you're on.

Friday, March 11, 2011

Projects You Didn't Know About

One of the fun parts of support is that often you don't hear about a project a client is doing until they're in the middle of it.... and they have a problem or a question.

So you'll get an innocent-sounding question: "Is the OtherIdType XML node really required?" A quick check of the cc list on the email will show another vendor's name on there.

Congratulations. You've just entered a project you didn't know existed. Oh, and there's probably something not quite smooth about it (you may never hear about the projects that go smoothly). Think fast - you're standing in some metaphorical quicksand.

At this point, you have two goals:
  1. Answer the question
  2. Figure out what's going on to make sure the whole thing isn't sinking
Answering the question is important; you should pretty much always try to help your client understand your system better.

More important, though, is helping the overall success of the project. If you're standing in quicksand, not only do you want to stop the immediate sinking, but you also want to get out of the quicksand pit - or you're just going to start sinking again!

So answer the question, but more importantly, call your client (and not the vendor), and ask them for a 5 minute overview of the project so you can answer any other questions they might have. It takes 10 minutes, but that phone call can prevent a lot of future headaches.

Monday, March 7, 2011


I got a requirement from a customer recently that said this among other things:

"The report should be the same"

Okay, great! Same as...... what, exactly?

I feel like this is one of the easiest requirements problems to find. It's simple:

Every comparison should have a comparator.

If you're going to say something is "the same", you'd better say what it's the same as. If you're going to say something is "different", you should probably mention different from what.

When I get requirements, this is one of the first things I look for, because it's really fast to find, and it's a good indicator that they haven't really explained much, at least in the information you're looking at. These requirements usually imply that you've crawled inside the head of whoever wrote them and are happily gathering your requirements directly from their thoughts. As soon as this becomes possible, I'll be happy to. Until then, I'm going to kick these back to you and ask for a comparator for every comparison.

Friday, March 4, 2011

Good Change Is Not Bad

When we change things, we change them for ourselves, but we also change them for everyone around us. When an engineering team, for example, decides to change from a waterfall methodology to an agile ship-every-two-weeks-and-ask-the-customers-to-participate-frequently methodology, that is a huge change for the engineering team. How that engineering team builds software, thinks about requirements, performs testing: all of these things change.

Turns out that "engineering development methodology" change doesn't just affect engineering. It affects many other people:
  • The support team now has to figure out how to insert escalations into the backlog and set expectations with customers around fix times differently.
  • The requirements team now has to create small requirements frequently for the backlog instead of large requirements documents infrequently.
  • The customer now has to figure out how often to upgrade and how to handle the additional workload of providing feedback and answering questions.
None of these changes is inherently bad, but they are all changes. Frankly, at least some groups will probably like the new way better. However, these groups didn't initiate the change. Rather, the

The problem with a change like this is that most of the affected people didn't initiate the change. Rather, they had change thrust upon them. That's scary.

A change cannot be good until we believe that the change is not bad.

Before the affected group can see the benefits of the change, they have to see that the change doesn't hurt them. Benefits here are benefits for the receiver, not benefits for the initiator. After all, the initiating group wouldn't have made a change if they didn't think it would help them!

The lesson here is that preaching the benefits of change will fall on deaf ears. First, overcome the fear - show that the change doesn't cause harm. Do this explicitly. Don't say, "This is awesome for you because.....!" Instead, provide comfort first without discussing the benefits: "You'll still get your X that's important to you through Y means." That way the receiver doesn't feel like the initiator is trying to hide something scary behind so-called benefits. Instead, the initiator is addressing the change and the fear directly.

Address fear of change. Then address benefit of change. In the end, you really can have both!

Tuesday, March 1, 2011

Frustrations and Rants

The vast majority of the time, we can all get along and still get work done. We work together and frequently agree with each other.

And then there are the few times where that doesn't happen.

Work is fun, but sometimes it's really frustrating. Sometimes people take that frustration out on each other. It sucks, but it happens. And you find yourself on the giving - or receiving - end of an attack. A rant that's about frustration, but that dredges up every disagreement along the way.

Whoo.... deep breath.

First of all, sometimes frustration is warranted. Sometimes you really are in a situation where only a sharp rebuke will bring the other party to her senses. For example, I worked with someone years ago who consistently did a large checkin, broke the build, and went home for the night. Her response every time was, "whoops. I'll have to try not to do that again." That was acceptable the first and second time. By the sixth time in a month, it was time to express the team's frustration.

But no matter how good a rant feels, hold off. Slow down for just a second. Rants are costly - that foaming-at-the-mouth-sputtering look is not flattering. You don't want to have to do it often. Make your rant effective.

Frustration will only be effective if it's expressed accurately, sparingly, and with purpose.

Frustration can be a powerful communication tool; it's a strong wake up call to someone to say, "hey, there's something really wrong here and it's important that you fix it.". That only works if frustration is expressed:
  • Accurately. Accuracy serves two purposes: doing the research may change your perception of the problem; and you keep the focus on the effects of the behavior rather than getting sidetracked into the details of the behavior.
  • Sparingly. If you're the guy who snaps at every little thing, then your snap loses power quickly. No one cares, because, hey, that guy whines when the free pizza has an odd number of pepperoni on it. If you always snap, all your problems are at the level of the odd pepperoni count.
  • With Purpose. Have a goal in mind. The purpose of a rant or frustration expression is to achieve change in behavior. To ask for that change, you need to know why you're expressing frustration. Also, if you don't have a goal, you probably shouldn't start ranting because it's likely to be misdirected.
So how do we do this?

Take the build-breaker I used to work with. Once we decided to approach her, and once it became apparent that "please don't do that again." wasn't working, then how we expressed the team's frustration had a huge effect on how likely we were to see her change.

We could say:
"Okay, seriously?! You always break the build and then you leave us to fix it! What do you think we are, your minions? Grow up, already! Be a real engineer! Have a little pride!"
It would feel good to say. It's also inaccurate. The word "always" is a big hint that we're probably inaccurate. It also fails to ask for change; we don't describe the behavior we're seeking. Lastly, we're attacking the person, not the behavior. This is a rant just about designed to make the recipient defensive and to escalate the disagreement. Let's not say that.

Instead we say:
"We've noticed a pattern where 6 times in the last 4 weeks, you've gone home at night after a checkin and the build is broken. That leaves the rest of us to fix it, on top of the work we've committed to, and we're getting pretty frustrated that this is becoming a pattern. What needs to change so that you can make sure your checkins finish building before you go home for the night?" This doesn't feel nearly as good to say; it's not nearly as satisfying as unleashing your wrath. But it's a lot more likely to be effective. We've done our research and we have an accurate description of the problem. We're noting our frustration and that there is a problem that needs to be resolve. We're ending with an action call: asking her for an immediate response targeted at a fix. We're leaving the solution itself up to her; that way she has the opportunity to identify a fix that resolves the problem while still meeting her needs.

Finally, by approaching the frustration gently and with a tone of reason, we've left ourselves an escalation path. If the problem continues, we can impose a stronger solution, or even indulge in a rant. We hope it doesn't get there, and that the problem is fixed while preserving the working relationship. But if it does get to a new and more frustrating point, we've left ourselves the escalation path to handle that.

Frustration happens, and sometimes we need to act on it. That's never an excuse for a rant, though. Solve the problem. Express the frustration. But be nice about it.