Friday, March 30, 2012

Fewer Is Sometimes Better

When I'm in a group debugging something or testing something, there's a tendency to start throwing out ideas.

For example, "Hey, guys, I'm noticing that writing to the system is really slow."
"It could be the disk is getting full."
"Maybe there's a network problem - bad cable or bad port."
"Perhaps we're saturating the network link."
"Well, we could just be reaching a really really big index and searching it is slowing down."
"What about memory? Are we swapping?"

We have a lot of ideas, a lot of possibilities, a lot of guesses. Sometimes that's okay. Sometimes you really do need to generate a lot of ideas.

Much of the time, though, we should probably do a little more thinking and a little less brainstorming. Sometimes with additional thought or analysis, we can identify one or two causes that are the most likely, and not end up wasting time chasing scenarios that are implausible as soon as you start to look at them.

Take our example above. Several of the suggestions are network-based. A little thinking could say something like, "well, for each item written, we send X bytes over the network interfaces. At the fast Y rate, that's about 10KB/sec, which is well under the network speed. So at the slow rate we're almost certainly not saturating the network." We can use similar analysis to eliminate index size, disk use or memory as likely candidates based on the total amount of data in the system. All of a sudden, we're down to one or two likely candidates rather than five or six. Our debugging just got a lot more focused.

Brainstorming is good when there are a lot of possibilities about what's wrong, when it's hard to eliminate any possibility, and when there is a large team that can analyze independently. In other cases, brainstorming might not be effective. When the team is small, time is tight, or you're in an interview, take those few extra minutes to think about each idea. Fewer is sometimes better.

Tuesday, March 27, 2012

Where You're Insecure

I spend s fair amount of time talking to people. Usually when I start a new gig, I'm there because some change is desired. There might be a problem, or maybe we want to build the next part of the big dream, or there's a process problem, or there's a need for growth. My first step when I come in is to shut up and listen. I talk to everyone who will spend time with me (donuts make good bribes). And I learn a lot, both about the thing being changed and about the people involved. Many people are looking forward to change. They want to solve the problem, or build something cool. And some people are really defensive. I walk away and I think to myself: Holy crow, this guy was defensive. But why? I actually don't know. I don't know this guy at all, so I don't know what's in his head. But I can guess. He's worried because it's an area where there might be a problem. Most of the time when I run across someone who is defensive about something, it's because they have enough fear about that thing to think there's a real problem. And it's often a fear based on some evidence. And THAT means it's useful. For example, I was working with a new team, and I had been brought in to analyze their content production process. So I sat down with all sorts of people who were involve, and I asked them about the process. I got a range of responses. The writers mostly thought it was fine; they liked the current process because it was easier. The editors hated it, since they were the ones who had to keep writers and drafts in sync, and they spent a lot of time copying content from one system or another. And then I ran into the graphics guy, and he was defensive. He thought everything was just fine thank you very much. And he didn't think any changes at all would be a good idea ever. It took some diving, but eventually I learned that the graphics guy was a pretty good layout person, and he was scared that any change in the process wouldn't let him tweak the layout. In other words, he was scared that a new process would expose a hole that already existed. And that was really good information to find out, because he was right. The new process we were considering didn't account for that at all. So when someone's defensive, dig a little deeper. There's likely to be a good reason why.

Friday, March 23, 2012

I'm Spartacus

When people ask what I do, I usually say, "I'm a software engineer". When I'm talking to someone who is not in software, it's a lot easier to say that than to explain that I'm a consulting VP of Engineering. It's also true. I work in software and I do work that - at least in the U.S. - is described as engineering. That is, it's technical work with the purpose of creating and shipping software.

And for all the time we spend bickering between engineering groups or teams, we're all still software engineers. It's not about our name. It's not about are role. It's not about how we got here or what degree we hold.

We're all here. We're all banding together with a purpose. We are all Spartacus.


Just check it out. We do stuff! We're (software) Spartacus!

Wednesday, March 21, 2012

The Madness of March

One of my clients is doing an NCAA March Madness bracket. Most people in engineering are doing it, and they sent me an invite, too.

An Aside:
This is a really good thing, as a consultant. It shows they have forgotten about the employee-consultant divide. You're just one of the team, which is pretty much exactly where I want to be.
End Aside.

I filled in my bracket, and it's been interesting. You see, I don't actually know anything about basketball. So I went with the highly scientific heuristic as follows:

  1. schools where I've heard of the basketball team
  2. schools I've heard of
  3. wild guesses
As of right now, I'm winning. (yay?)


I'm finding this really interesting. By rights, I really shouldn't be winning. Some of the people here did a whole lot of statistical analysis, and follow NCAA basketball very closely; they're far more informed than I am. They also took a lot longer to put together the bracket.

I suspect there's a lesson in there.

Maybe, just maybe, when faced with a highly variable system, it's as good to go with a simple heuristic as it is to create a complicated solution to the problem.

You don't always have to account for everything. As long as "close enough" or "better than anything else" is an option, choose the simple thing. It might just work for you.

Monday, March 19, 2012

Actually, Failure Is An Option

Ahhh the classic phrase: "Failure is not an option." Uttered by entrepreneurs, Hollywood generals (I don't think real-life generals use the phrase much), and desperate product managers around the world, it's as much a part of corporate ritual as any phrase. The only problem is that it's completely wrong.

Failure is always an option.

Any time we engage in an endeavor, there's a chance we might fail. Sometimes the chance of failure is very slim and sometimes it's very high, but there's always a chance. For example, later today I'm going to meet with a client. There's a chance I might not make it to the meeting place. I might get lost, or get hit by a bus, or have a heart attack on the way. It's extremely unlikely, but it's possible that I won't make the meeting. Fortunately, I didn't say "failure is not an option!" before I headed out.

So what do we really mean when we say, "Failure is not an option"?

We mean we really really don't want to fail. We mean that the consequences of failure are really very bad and we don't want to experience them.

For some people, this is when you're facing a major inflection point. Think our Hollywood general getting ready to defend the last redoubt of the good guys from the terrible evil enemy. If he fails, then all that is good in the world will be gone forever and we'll be alien slaves (hey, it's Hollywood!). Or, in the realm of software, maybe this is closing a deal that's going to keep the company from going out of business. Or maybe it's releasing a new product that keeps your division open and keeps everyone from getting laid off. In all of these cases, we might actually fail. And it would really not be fun. But it is an option.

For other people, "failure is not an option" is something of a verbal tic. Everything is of monumental importance to them, and they're failing to consider actual consequences. These are the same people who can look at a list and make 70% of the items on that list be top priority. These people have to be managed very actively, or their panic can bring down the productivity of an entire team.

When someone tells you, "failure is not an option", that's time to have a conversation about why this is so important and what we can all do to make failure less likely. Then take a few minutes to contemplate failing, consider the consequences, and allow for them. We don't want them to happen, but it's best to be prepared. After all, failure is always an option.

Wednesday, March 14, 2012

Virtues of Error Messages

In my role as a consultant, I frequently find myself picking up unfamiliar technology. Sometimes it's a new framework that a client happens to be using. Frequently it's an application or code base I'm evaluating on behalf of a client. Sometimes it's a bake-off between products my client is considering to solve some purpose.

One of the most valuable things with getting up and running with these unfamiliar technologies is this:


The error messages.

With an unfamiliar technology there's a good chance I'm going to do something wrong when I first get started. Usually it's something like a missing dependency, or a configuration that needs to be set up. These aren't major problems, but they sure can seem mysterious. After all, I'm often working with someone else's application; it's not exactly the same as doing a "hello world" on [insert framework of choice here].

This is where error messages come in. A good error message will point toward what's wrong. For example, "missing config file in app/config/database.cfg" is a good error message. A bad error message will at least tell you that something is wrong. The worst is when there is NO error message.

For example, I was starting a grails app the other day, and here's what happened:


Running Grails application..

And then the command prompt.

No error message. No indication there was something wrong. Just.... nothing. Oh I eventually figured out that there was a problem (no app running, nothing listening on any new ports), and then I figured out what the problem was. But having an error message - even a bad one! - would have gone a long way toward figuring out what happens.

So please, if you're writing software, make sure you write in an error message. It doesn't even have to be good. Just don't ever fail silently. Your fellow software engineers will thank you.

Monday, March 12, 2012

More Testable

I got into an argument this weekend about making code more testable. It was actually really interesting, because my defense maybe wasn't as good as I'd thought!

Here's the setup:
We were talking about some code that has an external dependency built very deeply into it. Specifically, it makes a call out to a third-party library right in the middle of a few methods. He looked at it and said, "Not a great idea, but it's working. Maybe we'll fix it eventually." I looked at it and said, "ouch! That's really not very testable. We should make changing it a bit of a priority."

And the argument:
Him: "It's working. I agree it's not ideal, but why does that mean we should change it urgently?"
Me: "It's not testable as it. I can't really exercise what happens if this third-party thing has problems or goes away."
Him: "So what?"
Me: "Umm.... err...."

I eventually was able to articulate that as unlikely as it was that this particular dependency would go away or return something unexpected, it was possible and I'd like to test for it. And he agreed that would be nice. And we agreed that... you know what? It's really not that urgent.

Nevertheless, I learned something.

Testable is not itself a goal. It's a means to an end, but if you can't articulate the end, you probably don't need the means.

And that's true of all the "able"s: testable, usable, maintainable. Unless it provides actual value to someone, you don't inherently need it. Usability is nice. Maintainability is nice. Testable is nice. None of them are in themselves overriding requirements. Rather, they are exactly as valuable as the value they provide to the consumer of the software (internal or external). Make your justification about the customer; it's much more compelling.

Wednesday, March 7, 2012

"The Business"

One of the memes going around these days is the idea of "the business." "Software engineering is about serving the business." "I don't know, let's see what the business wants." It's a contrast point: engineering is the people sitting in the darkness producing product; and the business is people who know what should be done and who bring in the revenue.

Now, I hear a bunch of you protesting that engineering isn't monolithic. We have developers and testers and operations people and managers! Guess what? The business isn't monolithic, either. You can't actually ask "the business" for advice. You can't expect guidance from "the business." You can get guidance from sales, or from marketing. You can get your paycheck from finance or HR (yes, please!). You can tell upper management that there is risk in some area.

So what do we really mean when we say that engineering serves the business?

We mean that the company isn't about software, even if it's a software company. The company is about producing revenue by providing a product or solution that fills a market need.

Everything we do is ultimately pointed toward that goal. And that goal is driven not by engineering but by the people who understand the market need and the benefits that our solution provides to fill that need. That means marketing and sales - people who are out there looking at the market and talking to customers - are the ones who directly achieve the company's goal. Engineering's job is to actually build the product or solution. And in that sense, engineering should be listening to other parts of the company.

We're just starting to see a wave of thought in engineering about working effectively with the rest of the company. For now, we can call that "the business." But keep in mind the ultimate goal: fill a need and make some money doing it. Listen to and work with the people who know how to achieve that goal - engineering will be better for it.

Friday, March 2, 2012

Divide and Conquer Planning

I work with a client who plans three-week sprints. They've recently started taking a divide and conquer approach to planning. It goes something like this:
1. retrospective, all together as one team
2. split up the candidate stories and assign to two or three groups
3. split into groups and plan

But but but! This isn't real scrum!

No, it's not. But it works for them. It's taken a planning process from a painful two days per sprint to a semi-painful 5 hours per sprint. The gain mostly comes from the team being so large - it's about 15 people. That's already about twice the size that many agile practitioners will advocate.

Here's a hint: if you're already violating some of the rules of whatever process you're following, then don't get all stuffy about violating other parts of the rules.

The goal of process is to do what works. If that means you work with a large team, okay. If that means the team trusts each other enough to take their plan, okay.

In other words, be strict or don't. Either way, don't sweat it.