Why I Haven't Been Hiring Testers

As many of you know, my engineering career started in software testing. I'm in engineering management now, and spend most of my time building and managing teams, writing code (product code and test code), and handling technology strategy for various clients. I feel like that's important to mention up front: I am not anti-test.

But.

Well, here's the thing. I haven't hired a straight up manual tester since 2007. I haven't hired anyone to a test position since 2009. I've hired developers, contracted designers, even hired project managers and contracted documentation experts. But no testers. I know what kind of value a tester can bring to an organization... so why haven't I hired one?

Well, partly because the testers that we did hire mostly stayed. That guy I hired in 2007 is still with the company and still doing testing. The guy I hired in 2009 is still with the company, although he has moved into development. We're still getting the testing function.

There's another part, too. I've basically stopped hiring testers as an early team component.

The testing function is hugely important. We do want to know that what we built works and that it fits our customers needs. So why no testers?

Increasingly, I have other people who are embracing the tasks that used to be reserved for a dedicated tester:
- developers have survived - thrived even - while writing test automation
- product owners have been eager to use the product in a structured way
- customer service reps and implementation managers have been able to provide additional context about application usage and behavior
- improved monitoring points out problems in dev, QA and production, and makes the patterns behind them apparent so debugging is a lot easier

Now, for some applications I'd still hire a tester. I'd make that hire when I needed that kind of specialized knowledge. But that knowledge is becoming less and less specialized as more people start listening to the mantra that "quality is everyone's responsibility". We're seeing more people who test, even as I'm around fewer testers.

Test is not dead. Test is more pervasive and discussed than it has been for most of my career. The dedicated tester, however, is becoming more rare. And I think that's actually good.

The Guess the Language Game

As I'm surfing the net, or riding the subway, I notice ads that contain code. Usually they're ads for job sites or degree programs. Sitting on the subway is actually pretty dull, so I find myself playing that "what language is that?" game.

For example, this is an ad I saw today:

This one was for a job board (job boards play this game a lot). Other common advertisers are training or college programs; and tech or biotech companies. It's a fun mix of languages and styles. Here are some recent ones:

• a for-profit college: Java
• biotech company: C++? (I think?)
• a large college: Java
• the same large college: JavaScript
• a job board: that was totally not a language. We'll be polite and say pseudocode
• another job board: JavaScript
• a technology company: Perl

The use of actual recognizable languages is surprisingly high!

It's also interesting to watch how languages change. When I first started noticing these ads four or five years ago, the language was predominantly Java or Perl. These days I see a whole lot of JavaScript. I don't necessarily think that it says anything about languages you need to know, but it sure makes a subway ride more fun!

The Hard Way Gets Easier

There are two ways software development and deployment can go: the easy way; and the hard way. When it's the easy way, code works, deployments are one-click (or one command), and unicorns dance through rainbows during each build. This way doesn't happen very often outside of tutorials and extremely small projects. No, all too often, it's the hard way.

The hard way is characterized by messiness. Maybe the code is sloppy. Maybe the code is great but the deployment process is manual and error prone. Maybe the code and deployment ideas are great but different environments are all hand built, and none quite the same way. Maybe everything's fine, but getting approval takes forever so releases are slow. Maybe the development process requires things that don't get done, so you're stuck waiting on a code review... for days. Either way, nothing ever goes as smoothly as we'd hope. To some degree, "hard" characterizes most software development shops I've seen.

(Tangential note: I'm a consulting engineering manager. I probably see a higher proportion of software shops doing things the hard way. If it's already easy then my services are generally not needed!)

Building software the hard way isn't fun. It's frustrating for engineering, most of whom know better but don't have a clear path (or don't have time) to get from current to easy. It's frustrating for management, who would like to see new features more quickly and more reliably. I know very few people who actually enjoy doing things the hard way.

So how do we break the cycle? How do we make it less hard? We fix it. But we do so slowly. The first mistake most people make when they realize that developing and delivering software has gotten hard is that they try to fix it. All of it. Right now. That's a recipe for failure. If they can't fix it all at once - and they can't - then they don't fix any of it.

The trick is to make small steps toward a fix. Management isn't going to approve stopping all new work for three to six months and completely rebuilding production just to make it easy. So we do things that management can approve. Remember, it's in management's interests to see this problem solved, too; they want to approve changes that will make things easier. They just have to balance those changes with meeting all the other obligations. Hence: baby steps.

Start with finding something small you can change. It doesn't have to solve the hardest thing. It doesn't have to address the biggest problem. It doesn't have to fix things all the way. Our only two criteria are: (1) we can do it; and (2) it makes things a little bit better. Then do that one small thing.

You can guess what comes next. Yup, do it again!

Doing things the hard way is an uncomfortable reality for many of us. We can make it easier, though. Think small, think doable, and eventually what's hard will get easy.

The "Hardening" Myth

The purpose of software changes as it ages. When we first write something, we're doing it quick and dirty, trying to validate that the core idea is good. We might call it a prototype, a proof of concept, and early alpha, beta, or even 1.0, but the commonality remains: the software has been written with speed and  getting the core of the offering right. This is true whether the core is an algorithm, business process, market hold, whatever.

Once we get through that first phase, though, it's time to build "real software". This is the time you build the software you can grow on. All those things that got skipped in the first phase - error handling, monitoring, administration - need to get put into the product.

With product life cycles being what they are, that frequently turns into "hardening" the prototype. Basically, engineering is going to go off and add administration, and configuration, and error management, and scaling capabilities. Engineering probably says they want to rewrite at least part of the system, but gets pushed (or walks themselves) into the idea of hardening.

Bad idea alert!

"Hardening" is building something that looks like this:

Looks nice on top. Pretty shaky down below.

"Hardening", you see, is a myth. Way back when the team first built the prototype, they either "built it right" or they didn't. They either set up robust deployment, included monitoring, got their configuration and logs into a good location, or they didn't. If they did, then you're not having the hardening versus rewrite conversation at all. If they didn't, then you need to do a rewrite.

"Hardening" is a compromise that leaves you with baggage, slowing you down over time. The symptoms vary. Maybe you have a production system that you can't shut down because no one knows how it was deployed but it's functioning (for now). Maybe your releases are high risk because you have to change configurations in code. Maybe you don't have monitoring, so you only know about downtimes when your customers call (whoops!). Maybe the system has a lot of errors, and consumers are confused, creating a burden on your support or dev team. Whatever your particular symptoms, you're seeing the effect of trying to take a prototype (or alpha) and make it into software that will serve you at scale.

There's no shame in rewriting software. It's not a bad thing to throw out software once it has accomplished its purpose. If you wrote a prototype and it showed that your core algorithm worked, great! It has done what it needed to do. Time to retire it and write the software that will implement your core algorithm at scale.

So don't "harden".  If you can proceed with what you've got, then great. If you can't, bite the bullet and rewrite. It'll slow you down in the immediate term, but it's the right path to long-term velocity.

Test Intent or Implementation

Writing unit tests for existing code can be very simple. You pull up the method (or API or class or whatever), look at the implementation, and start writing tests to make sure you hit it all. Read method, write test, read method, write test. Repeat until desired test coverage is achieved.

Yay! We're done!

Well, no, we're not.

We've tested what was implemented, that's true.

But did we test what was meant to happen? Did we test the intent?

This is a much trickier question. If we're testing a method that's entirely internal - a helper method, say - then the intent and the implementation are probably identical. After all, the consumer of the method is the guy who wrote it, and if it doesn't do everything he wants, then he's probably noticed by now! If, however, we're testing something that is consumed externally - an API, say, or a library - then the intent may be quite different from what the programmer understood.

For example, let's say we have an API that provides some information. The programmer may have implemented that as a GET with url parameters for each data point (e.g., http://mycoolapi?id=1&name=bob&text=hi). The implementation may be perfectly correct. But the intent may be quite different. The product manager may have meant for the API to be a POST with a multi-part form upload, since the next data point is going to be a picture that can be uploaded. Implementation - perfect! Intent - oooops.

So when you're testing something, go ahead and look at the implementation. Just don't forget to look at the intent, too!

The Myth of the Passionate Employee

I've had conversations with two separate people in the past week, and in each the person I was talking to said, "No, I really need an employee. I want someone who's going to be passionate!"

It piqued my interest. When you really sit down and parse those two simple sentences, there are several assumptions and definitions in there that are fascinating.

First there's the notion that an employee - rather than a contractor - is needed to display passion. I'm not sure I buy this, although I can see where they're coming from. After all, if you pay a contractor to build something, they're going to do the job and expect to be paid for it, and that's all. If they're good at what they do, they will do a good job and do what it takes to deliver quality work on time. An employee is theoretically more closely tied to the future of the company and therefore will display more passion about it. Unfortunately, this is a loose correlation at best. There are passionless employees and there are contractors who are highly passionate about each of their clients.

Second there's the definition of "passion." I do not think this word means what  my conversation partners think it means. After all, passion merely means "strong emotion". What they think is means is more along the lines of "cares deeply about the future and vision of the company and will work as hard as it takes to see that vision come true." And that is a very different idea. Both of the people I was talking to are company founders. They live and breathe these companies: they stand beside their sons' soccer games thinking about closing the next client; they send emails - and reply to them - at all hours of the day and night; and they've sacrificed normal jobs for this. They want everyone to want their companies to succeed as much as they do, and to work as hard. That's what they call "passion".

Here's a little hint: It ain't gonna happen.

There are people who work very hard for dreams. They do it because they personally are getting something out of it. Maybe it's a chance to be rich ("I'm gonna be the next Google millionaire because I'm employee number 2!). Maybe it's advancement in their career. Maybe it's fame (the guy who came up with Amazon AWS is a rock star in some circles). Maybe it's just because they're really interested in the work.

There are other people who won't work that hard. Your dreams and blood and sweat and tears are their job. They want to do their job, get compensated fairly for it, and go do something else.

Specifying an employee or a contractor won't guarantee you passion or lack thereof. The financial arrangements of the work - employment or contracting - really don't have anything to do with how hard they work and what they want for it.

Working hard or long is also no guarantee of quality. Ask yourself what you really need. Do you really want someone who will be there all the time responding to your emails and putting in the hours - regardless of how good the output is? Or do you want someone who is going to help you build your product well, even if that someone will only do it in 40 hours a week?

So the next time you say you want a passionate employee to work for you, don't. Step back and think about what you really mean. Do you want someone who will build good things? Or do you want someone who is always around? Me, I'll take building things.

Your choice.

A Rant On Learned Specialization

I've written a few times about specialization in various forms. It's a large point of consideration on small software teams. On the one hand, having a specialist on hand means that the work can get done generally more quickly and better. On the other hand, having a specialist means that when he's not available that kind of work just plain doesn't happen.

Ultimately, we as engineering teams generally have a few fairly concrete goals:

• deliver functionality to our customers as fast as possible
• don't kill ourselves - we make mistakes when we work too fast
• create solid software so that it can survive the onslaught of usage (yay success!)
• provide management with predictable production rates - they need to know when things are likely to happen, even if it's just an estimate. Most of the time we call this velocity.

None of that says anything about specialist or generalist, at least on the surface. If we have specialists, some parts of our goal become easier; we are more likely to create solid software, and we'll probably do it a bit faster. If we have generalists, though, we're more likely to be predictable; one person being gone doesn't stop everyone else from finishing things, and we have fewer bottlenecks since anyone can jump in to help.

Specialization is occasionally necessary. If you're doing something truly specialized - a new data storage technique, or high volume (read: millions per second) data processing - then you need a specialist. Or two or three. For most of us, though, we're not doing anything that specialized. We're building web applications, or apps for various devices, or installable software. We have to be good software engineers, but the problems aren't really that esoteric. Even in cases where the product is hugely specialized, there is still a large part of it that's just a solid program to do the more commonly needed parts (administration, logging, UI, etc.).

It's a lot like physicians. Most of the time what you need is a general practitioner who is going to know how to solve lots of different kinds of problems that many people need solved (chicken pox, colds, appendicitis). You really only need a specialist when something truly weird is going on - Wilson's Disease, for example.

Here's the rant part:

There is another, more insidious, form of specialization that I like to call "learned specialization". This is where a person or a team who could be a generalist decides to become a specialist. This is what's happening when you hear statements like, "I'm the architect. Therefore I have to do all of the design and you can't start that feature yet because I haven't designed it." Or you hear things like, "Well, Jeff should do that because he knows the frobble module." Or "I'd really rather not tackle that story because I haven't worked in the admin before." The team - or one person on the team - is creating unnecessary specialization. There's no reason that John can't learn the frobble module, even if Jeff already knows it. There's no reason that Jorge can't produce a design for a feature and get the architect to take a look (must assuage that ego!).

The worst part of learned specialization is that it's self-reinforcing. The longer that Jeff works on the frobble module, the better he'll know it and the more other team members will just avoid those tasks because Jeff is slowly getting faster and better at working with it. The more Janet avoids the admin module because she doesn't know it, the larger it will get and the more there will be that she doesn't know. If the architect does all of the design, no other team member is going to learn how to design software effectively.

Be vigilant against learned specialization. Any time you see incipient specialization occurring, take it as a training opportunity and get someone else in there. Ultimately, long term software success is about breaking down silos, creating an environment in which there are multiple points of redundancy, and encouraging an atmosphere of training rather than avoidance and burnout. You'll only get there with generalization.

Positive Negative Assertions

I've been working with someone learning how to test, and we ran across our first negative test case. The method we were testing looked something like this:

new_car_color = paint_car(color)

You feed it the color you want your car, and it returns the color of the new car. If you feed it nothing, it returns the current color.

We'd been writing positive tests that looked something like this:

def test_paint_new_color
   color_i_want = "red"
   color_i_got = paint_car("red")
   color_i_want.should == color_i_got
end

def test_dont_paint
  color_i_want = "blue"
  color_i_got = paint_car("blue")
  color_i_want.should == color_i_got
  color_i_got = paint_car()
  color_i_want.should == color_i_got
end

So when we got to the first negative test, we talked about it and came up with this: "We want to test what happens when we try to paint a car a color that doesn't exist".

The junior tester then wrote up the following test:

def test_paint_no_color
    color_i_want = "not a color"
    color_i_got = paint_car("not a color")
    color_i_want.should != color_i_got
end

Well, it's a start. But our car could be any color as long as it's a color. Is that really what we want? What if we started red and the method painted the car blue whenever it got a color it didn't understand? We wouldn't have a clue!

Instead, we write the method like this:

def test_paint_no_color

  current_color = paint_car()

  color_i_got = paint_car("not a color")

  color_i_got.should == current_color

end

We make a positive assertion and confirm that the color of the car didn't change.

Even when we're doing negative tests, our assertions should be as positive as possible. We should assert that something is rather than asserting that something is not. After all there is a large range of things that fits inside "is not" - and we might not like all of them! There is a much smaller range of things that fits inside "is". So wherever possible, we should do positive assertions, even when we're testing a negative case.

The Game Fad

There is a fad wandering around the software community these days: games. In particular, logic games like SET are becoming very popular in training exercises, interviews, conferences, etc. The basic idea is that playing these games can teach you to be a better tester, or developer, or product manager. Someone who is good at SET can readily see patterns, can show flexibility (use one card in multiple sets), and can make logical deductions. These are all characteristics of a good software engineer. So does this work? Or is this just the next version of the "how many piano tuners are in the U.S.?" questions that were so popular in the 1990s and early 2000s? In the spirit of fair disclosure, I should mention that SET Enterprises is a client. So in my particular case, understanding SET is actually helpful for my job. For most people, though, games are a proxy for the actual work. They're an analogy come to life. Most of us don't build, test, and maintain games. The trouble with software is that it is highly abstract, and so it's hard to really know that any two people understand things the same way. It's also hard to distinguish familiarity with something from "read an article about it", simply because usage of the same thing varies so much across different environments. For example, someone might say, "yes, we used factories" and be referring to test object creation with Factory Girl, and someone else might make the same statement and mean that they implemented the factory pattern. In addition, software engineering is context heavy, and context is less shared than in some other industries. If a bunch of lawyers or CPAs are talking, they can all know that they went through the same base knowledge - the bar or The CPA exam - and so they can assume some commonalities. There is no software equivalent of Generally Accepted Accounting Principles (GAAP). So in order for us to share ideas, we need to establish a baseline of knowledge and skills. When time is relatively short, say, in an interview, we seek out quicker proxies that highlight the skills we feel are relevant. Some of the best testers you will ever see are useless on their first day; they require in depth product knowledge for their particular skills to shine. We can't do that in an interview or at a one hour talk, so we find analogies that we believe highlight those trengths. We play games. So is a game a useful analogy? If I'm good at SET will I be a great tester? If I play a mean game of chess, does that say anything about my skills as a developer? Yes, but only if the skills are actually relevant to the kind of work I'm doing. The visual pattern matching in SET is great if I'm testing a UI centric application, or something where design aesthetic and precision are important. The logic and sequencing that a good chess player displays are great if the product I'm developing is, say, a finite state machine. However, if I'm testing an API, the visual patterns aren't so useful. If I'm writing a simple login module, then advanced logic skills are less important than other skills. Playing games is a lot of fun. So were those logic questions about why manholes are round. But before you put too much emphasis on them, ask yourself if they're really showing characteristics that are important for your particular situation. Then go play some games, if only for the fun of it.

Quick: RSpec::Core::Configuration::MustBeConfiguredBeforeExampleGroupsError

I upgraded my RSpec version to 2.7 in an old project today and immediately got this error:

arete:textaurant catherine$ rspec spec/*

/Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:471:in `assert_no_example_groups_defined': RSpec's mock_framework configuration option must be configured before any example groups are defined, but you have already defined a group. (RSpec::Core::Configuration::MustBeConfiguredBeforeExampleGroupsError)

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:168:in `mock_framework='

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:142:in `mock_with'

from /Users/catherine/Documents/turnstar/textaurant/spec/spec_helper.rb:25

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core.rb:71:in `configure'

from /Users/catherine/Documents/turnstar/textaurant/spec/spec_helper.rb:17

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/activesupport-3.0.10/lib/active_support/dependencies.rb:235:in `load'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/activesupport-3.0.10/lib/active_support/dependencies.rb:235:in `load'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/activesupport-3.0.10/lib/active_support/dependencies.rb:227:in `load_dependency'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/activesupport-3.0.10/lib/active_support/dependencies.rb:235:in `load'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:459:in `load_spec_files'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:459:in `map'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/configuration.rb:459:in `load_spec_files'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/command_line.rb:18:in `run'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/runner.rb:80:in `run_in_process'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/runner.rb:66:in `run'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/gems/rspec-core-2.7.1/lib/rspec/core/runner.rb:10:in `autorun'

from /Users/catherine/.rvm/gems/ruby-1.8.7-p352/bin/rspec:19

The solution is twofold:

1. Convert all old-style rspec includes (require File.dirname(__FILE__) + '/../spec_helper') to new style includes (require 'spec_helper').
2. Make sure that all your specs have " require 'spec_helper'", even the ones that don't have any specs in them. I had one spec file where all the specs were commented out (I know, I know! I'm fixing it!) and it happened to be loading first, and causing the above error.

Improve, Then Brag

One of the hallmarks of new (as in New Year), is ambition. This year, after all, will be different! This year we will be better, stronger, more efficient, and just all around great. Plus, we've all just gotten off a relatively slow time, maybe even had a vacation. It's time to get back to normal and to get back to work.

There's a temptation to take the zeal a bit too far. After all, the velocity points you got in December were slowed down by vacations, holidays, time off, etc. So we can increase the amount we think we'll do!

Just be careful that everyone on your team agrees.

It's fine to think you'll be better, stronger, faster; we all want to improve. But wait for some actual evidence of it before you go making claims based on being improved. See your velocity go up before you start bragging about those features that will be ready earlier than planned, or the extra features that will make it in. After all, not all improvements happen as fast as we want.

It's simple:

Improve first. Talk about it second.

Get those two backwards and you'll just depress your team and anger your customers by setting another set of unrealistic goals. So avoid unrealistic goals, don't make promises you aren't sure you can fit, and look for improvements before you assume they're out there.

Toolchain

I work for startups, mostly on web-based software and services (SaaS). Tools are everywhere, and we need to get a lot done. Fast. Our tools have to work for us, not against us. The right tool makes life a lot easier and gets features out the door faster. The wrong tool slows the whole team down, wastes money, and wrecks morale.

So what do I use?

Before I talk about specific tools, let me lay a bit of groundwork. When I'm working for a client, I frequently use their toolchain. When I'm running engineering, we frequently use the toolchain I describe below. Most of the projects I work on are web or mobile web applications, or software APIs (think SaaS).  We're almost always a distributed team, and we usually use some variation of agile methodologies (SCRUM-like or kanban-like). The important part to take away is that we're generally working on multiple things at once, we try to keep those things small, and we push new features or changes into production when they're ready, without waiting for a defined release cycle.

Feature Tracking:

Product: PivotalTracker

Cost/Level: $18/month will hold until your dev team passes 5 people, then $50/month

Summary: I'm not in love with this one. It works pretty well, but there are a few features I wish it had, like: (1) distinguishing between done and deployed; and (2) a UI that doesn't feel so darn squished.

Other candidates worth considering:

• Jira: Too heavy and too pricey. Annoying to configure, and really encourages excessive time spent on tracking ("ooh! pretty graphics!"). It's useful if you have nervous and overbearing management involved.
• Lighthouse: Good for very small projects with savvy business types. Good email and source code integration and very simple, but little to no workflow or enforcement of procedures.
• Trajectory: I've heard good things, but not used it. Somewhat niche.

Bug Tracking:

Just use the same thing you're using for feature tracking. Don't overthink it.

Source Code Management

Product: GitHub

Cost/Level: $12/month will hold until your dev team passes 5 people, then $22/month

Summary: It's hosted. It's stable. It's Git, which lets the dev team do amazing things.

Other candidates worth considering: Just use GitHub

Hosting

I can't make a single recommendation for hosting, since it depends so much on your product and your team. There are a couple of scenarios that might make sense:

• If your goal is cheap, you're on Rails, and you don't have a sysadmin type on staff, use Heroku.
• If your goal is reliability and you can do your own administration, use Amazon AWS and put your database on an RDS instance.

Product: Heroku

Cost/Level: varies based on usage, but the add-ons add up

Summary: This is cheap and it's the easiest scaling I've ever done. It's also well configured, which  makes a big difference. Big downsides: frequent downtimes (especially if you use the shared database), and it will only backup once an hour, so data loss is a possibility. Also, use the kumade gem if you're using heroku. The deployment model of heroku alone is pretty bad; kumade helps.

Product: Amazon AWS + RDS

Cost/Level: varies based on usage

Summary: This lets you start small and will scale with you. It's also very reliable (and can be made more reliable if you have more money to spend). Interaction with Amazon is a solved problem, so scaling and deployment are simple problems. Big downsides: you have to do your own configuration, which means you need some sysadmin skills to set it up.

Development Environment

This one varies based on your product and personal experience. I mostly work on web apps, and my preferences look like this: Ruby on Rails, TextMate (vim is fine, too), cucumber, rspec. As long as there ARE build and testing tools in place, let the dev team pick what they like.

Build and Deployment

This is another one based on product and personal preferences. Just make sure there IS one. Oh, and the deployment should be something that's run automatically from the build system. The answer here is a tool, not a manual process.

That's my stack. If I've forgotten an element, let me know and I'll be happy to talk about it. If you disagree or recommend a different tool, let me know. I'm always curious to see if there are better tools for my toolkit!

Always And Never Usually Aren't Either

One of the quirks of coming to software development through test and through business is the healthy fear it provides of absolutes. Things that should "never" be true or are "always" bad turn out to be true on rare occasions or sometimes beneficial. This holds in the software we write. For example, you should never  use the singleton pattern because it adds a hidden global dependency.... But a logger should can be a singleton because you really want to unify logging into one place that's configured by multiple things. This also holds for practices and procedures. For example, we should never use a test script.... But perhaps there is a situation where that would be useful, like in an automated smoke test where the script is in the form of code. There are no best practices at all ever... But I have yet to find anyone who thinks that keeping code in source control is a bad idea (maybe its a best practice!). Regardless of the topic within software, any time you hear an absolute - always or never - be suspicious. When someone talks in absolutes, there's usually a reason they're speaking with such exaggeration. Sometimes they're simply naive. Sometimes they're insecure and trying to hide it. Sometimes they're just seeking attention through dogmatic pronouncements. No matter the cause, be smarter than that. Notice the absolute statements, recognize that they say more about the speaker than about the statement, and move on. Things that seem absolute probably aren't.

Describe the Unusual

Documentation is a part of my job, like many engineers. I write documentation for end users, training material, documentation for other developers (API docs, anyone?), bug reports... all sorts of things. And like many documentation creators, I can talk for a long time about some of these things.

Saying too much is a very good way to frustrate your audience. They don't want to know everything; they can't absorb it all and they lose the important parts. So follow the cardinal rule of documentation:

Describe the unusual.

Let's assume for the moment that documentation is about describing interaction. It attempts to explain to the reader how to successfully work with the system - whether that interaction is through an API, a GUI, or whatever - and how to understand what the system is doing (i.e., error states, message meanings, etc).

Describing interactions is a huge thing, which is why it's so easy to write a whole lot of documentation. But that's the wrong thing to do. The truth is, most of the time you can assume some basic knowledge, and just describe what's different.

For example, I'm working right now on a logging module for a system. The system right now does ad hoc logging; each component logs in its own way. My logging module will provide centralized logging so we get consistency in log level, storage location, format, etc. When I'm documenting this, what can I assume, and what should I write about?

My audience is other developers who work for this company. They pretty much understand the system, the language, and general development concepts. Great!

So I describe the things that are unusual:

• instantiating this (custom) logger I wrote
• requirements around uniqueness and sharing of loggers (class instances)
• whether it is thread safe (yes) and process safe (no)
• deployment requirements and the configuration format, for the IT guy
• log size and rotation rules, which I got from the IT guy

I don't describe the things that developers generally expect from logging:

• the existence of log rotation and aging
• when to log at various levels (DEBUG vs INFO, etc)
• the format of the log entry

I can skip what people already know, which makes the documentation much shorter and makes the important parts more readily apparent. It's easier for the consumers to read and much more likely to be actually useful.

So when you have to document something, whether it's a multi-day tutorial or a simple comment describing how to use a method, focus on the things that make it unique. Describe the unusual.

Breathing Room

One of the fun parts about the end of the year is how many big initiatives tend to pile up. The culmination of a big project - whether that's a new website, a major refactoring, a release, or something else -  often results in a release. In many ways this makes sense. It is, after all, the end of the year, which is arbitrary but a really common breakpoint and time to assess accomplishments. It's also a relatively quiet time, with many people on vacations, so usage tends to be relatively light (except if you're retail!).

But.

(Ha! I always say "but".)

There's a catch. Big projects are fine, and putting them out at the end of the year makes sense in many settings. The risk is in trying to put out too many too quickly. Any big project comes with risk. When you do many of them at once, you compound your risk, and make it harder to figure out what broke when you're troubleshooting.

If at all possible, give your big projects some breathing room. Push them out and then go work on something small for a brief period. It doesn't have to be long - a few hours or so - but it'll make life a lot easier. Use the breathing period to:

• get a new baseline of system behavior and identify behaviors and patterns caused by your large change
• reflect on what you've done and tie up any loose ends
• review the next big change in light of the new system behavior, configuration, etc.; does anything need to change?

Breathing room is good for humans and good for systems. Take it, if you can.

Dependency and Redundancy

My software is, of course, perfect. (HA! but go with it). The trouble is that the software I write uses other software; I have external dependencies. For example, Textaurant uses Heroku for hosting, Twilio for text messaging, New Relic for monitoring, and Airbrake for error aggregation.

There are a several good reasons to have external dependencies like these, including:

• Development speed. Using providers for pieces of the solution lets us focus on the core value we provide rather than the common.
• Better reliability. People who specialize in monitoring, for example (like New Relic), are going to be a lot better at monitoring than someone who doesn't think about this all the time.

But.... there is a big downside.

When your external provider goes away, you're in big trouble. If Heroku goes down, my app is unavailable. If Twilio goes away, I won't be sending any text messages. It doesn't matter that the outage is on the service provider end - to my customers, it's just the application they use not doing what it's supposed to. And that's my problem.

So we have dependencies, which are really useful and also introduce risk. What can we do about it?

Lots.

Let's take a simple example. My hosting provider had an outage on December 3rd that took down our application. What could have prevented us from being unavailable, even as Heroku was having a No Good Very Bad Day? We could:

1. Add a second hosting provider, writing into a shared database (or db cluster). Properly load balanced, the hosting provider who was not affected could simply have taken over all hosting duties.
2. Fail over to a secondary hosting provider as soon as we realize we're down, again using a shared database or database cluster.
3. Use local data storage in the browser to allow users to keep working. It wouldn't provide full functionality, but it would have given us 85%+ of our features, which is a lot better than simply being down.

There are two common themes running through these options: redundancy and cost. We can increase redundancy.... as long as we're willing to pay for it. How far you go toward ensuring redundancy is tempered by how much time and money you're willing to spend. In the end, it's up to you and to your particular needs. Just consider your dependencies... before they go down and make you consider them in a panic!

Selective Vacation

It's never a good time to take time off. I'm pretty terrible at vacations in general. And yet, faithful readers will notice that I haven't written here in a week, which is a long time for me - that's practically a vacation!

One of the consequences of being a consultant and having several clients it that someone's almost always on a deadline. If client A just got through a big push (and is now taking a collective few days to decompress), that's frequently when client B is just facing down a big deadline. It makes scheduling vacations... tricky.

So I take selective vacations.

A selective vacation is a vacation... from part of my life. This past week it was a vacation from my blog. Sometimes it's a vacation from a client, or not starting a contract immediately. Sometimes it's a vacation from writing articles. Sometimes it's a vacation from cooking dinner! And sometimes it's a vacation from being home - I spent last week in California visiting family and working.

There are a lot of benefits to a vacation: a change in routine can spark new ideas; getting away can stave off any potential burnout; health benefits accrue from lowering stress on vacation; fun vacation photos or stories spark conversations with your coworkers. At least for me, vacations also provide an appreciation of routine (I do like my routine!). I get a lot of these benefits from a selective vacation - it sparks creativity, and gets me out of a routine temporarily. I also find it very relaxing - not only do I not have all the normal pressures and deadlines, but I'm doing enough work that I don't get stressed over missing important client calls or deadlines.

The point is that if you're like me and find it hard to stop everything and take a vacation, then don't. Take a selective vacation. It offers a lot of the same benefits of a real vacation, but it's a lot more doable.

Your Software's Personality

Software is a very cold and abstract thing, really. It's nothing we can touch or interact with. So we tend to anthropomorphize it. We think some software is cute, and other software is a workhorse, and still other software tries hard.

For example, Twitter is cute. It has fun little error messages. It features pastels, lots of colors, a good amount of white space, and cheerful graphics.

At the other end of the spectrum, Photoshop is a workhorse program. It has eight bazillion buttons and menus (I know, I counted), and it feels really powerful if you can just get it to do what you want. Of course, getting a program that big to do what you want is something like knowing how to use a workhorse - it takes practice!

As software engineers, designers, product managers, etc., we can help shape our software's personality. We can encourage users to think our software is fun, or simple, or dense and informative - all by the decisions we make while implementing it.

The first step is to decide what kind of personality we want. Are we building a consumer game that should be fun and irreverent? Are we building statistical analysis software where we want it to be prescient and nonintrusive? Are we building software for highly trained users, where we want it to be very consistent and provide hints without getting in the user's way?

Once we understand what our personality is, then we can find ways to express that personality through software. Consider:

• The tone of the text
• Graphics and colors
• Layout - how dense?
• The presence (and intrusiveness) of help and guidance
• the workflows - wizards or menus? how short or long?

Your users will give your software a personality - it's up to you to make it the personality you want.

On Tone

I was reading a blog post the other day that included a number of very similar comments. The blog post itself wasn't hugely important (feel free to read it here). The important part was that the blog post author was showing an implementation of something, and that implementation could have been simplified by using a different API call. Several of the comments pointed out the existence of this other API call.

This is all pretty simple, and quite common, but let's look at HOW they pointed out that the author's implementation could have been improved. Here are three comments that all say the same thing:

1. 



2. 

3. 

They take quite different tones, even though they're saying the same thing. Comment 1 is very short and neutral in tone. Comment 2 is quite aggressive, even belittling (or teasing, depending on how thick your skin is) the author. Comment 3 is much more gentle, posing feedback in the form of a question, even though the question almost certainly presumes an answer of "no advantage. I should use DictReader.". Question 3 is also the only one that provides a link to the referenced API call.

None of these comments is inherently better or worse than the other. Using the tone in comment 2 has the risk of making others think you're kind of a jerk. Using the brevity of comment 1 probably works best when it's safe to assume some level of knowledge (e.g., that the user can go find the docs for csv.DictReader). Comment 3 is the least likely to offend but the most likely to make the speaker look tentative or soft. The point is more that  you can express the same information many different ways. Take into account your relationship with the recipient of the comment and the type of comment, and use that to find an appropriate tone.

Cute On Occasion

There is a trend in consumer software - and to some extent in business software - to be cute. That can be a cute logo, or fun field naming in forms, or humorous error pages. It's all in good fun, and can frequently help personalize software. After all, software is kind of a remote, sometimes dull thing. Why not have some fun with it!?

I'm all for fun with it. I'm even all for fun with errors or error pages (see the Twitter fail whale for example).

But.

Be careful not to take cute too far. Cute is only fun when it's occasional. When it's frequent, cute just becomes frustrating.

So when you're going to do something cute and fun, that's great. Before you go for it, though, ask yourself: "how often will my users see this?". If the answer is "not very often", then go for it! If the answer is, "kind of a lot" or "every day", then don't be cute.

After all, it's all fun, until it isn't. Keep it fun.