Tuesday, June 30, 2009

What Makes a Good Tester

James Bach's comments on my post yesterday considering how a tester can align his perception of himself with reality got me to thinking. What makes a good tester?

The short answer is that I don't fully know. It will vary a lot based on the project, task, and my current team makeup. However, there are a few things I always look for, and a few red flags. The rest - and there is more - will depend on your project and your team.

So what do I think makes a good tester?
  • Curiosity. Testing is ultimately seeking information day in and day out. Someone who enjoys seeking out information is going to last a lot longer than someone else.
  • Detail sifting. A vaguely-worded bug, or feature, or overview of a release's current status and risk points is worse than useless; it looks like it provides information but it really masks some level of ignorance. Conversely, too much detail creates a problem where you can't see the actual information because there's just too much data in the way. A good tester can sift details for relevance and communicate them.
  • Translation abilities. Usually testers will wind up working with people having very different understandings of the system, different technical facility, and different needs out of a communication event (n.b., fancy speak for "conversation"). A good tester can quickly discern the other party's focus and skill set, and tailor the conversation to that.
  • Good memory. I can't count the number of bugs that I've seen found because, "hey, didn't we try that and see something funny last week?" A good tester will remember what I call the "niggling things"; all those little details that weren't important at the time, or that we haven't gotten back to investigating yet. Those niggling details are where your most interesting digging lies.
  • Logical. If I have to explain every detail of a system to someone, I'm going to go insane (and may take you with me!). I want someone who, given a pattern or a basic understanding, can go figure things out. Further, this tester has an ability to say "if X and Y, then possibly Z", even without the system actually being in place. This is invaluable in considering the ramifications of a feature change, or thinking about what may have changed in a system due to a configuration change, or considering where we may find bug clusters.
You'll notice I haven't said, "Finds a lot of bugs", or "Has a low duplicate find rate", or even "Can find the scary issues". Those things are side effects, not attributes of a person.

So there are lots of wonderful things to look for. What about the red flags?

There are certainly things that scare me:
  • "I always wanted to do this". Someone who "always" wanted to be a tester is already prone to exaggeration (at 5 you wanted to be a tester, not, say, a firefighter?). In rare cases that may be true, but it's more likely that you've got someone who's just trying to appear eager. And I'd rather have truth without exaggeration - what are you going to say when you talk about the system?
  • "I'm good at finding bugs". Okay, so you can pick things apart. Finding bugs is part of the job. What about the rest of it? Tell me about that, too.
  • Automation engineers. I expect all my testers to be able to approach a system with a test mindset. That includes those who will spend most of their time working on test infrastructure/scripts/automation/etc. You still have to be a tester. Note that for this one, I could imagine a situation where this wasn't a negative, but only in cases where I'm looking to fill a very specific position on a team that has the coverage in other areas.
This is, I suspect, a touchy area. I know that the things I value in a tester are not necessarily the only things - or even the same things - someone else might value in a tester. So let's hear it: what do you think makes a good tester?


  1. As someone who wrangles both testers and coders, I value diplomatic ability above all else. The clear documentation is nice, but even more importantly, they have to come across as being on the same side as my developers, helping to improve a product, and not as adversaries.

    It's tough, because good testers take pride in their work, and after seeing so many bugs over the years I think it's easy to develop a "oh cripes, not *that* issue again, why don't they ever get that right?" attitude. The potential for adversarial relationships is increased by the de facto pecking order whereby coders get more respect / pay / creative input than testers.

    The great testers I've worked with go out of their way to establiish rapport with the developers who see their issue reports.

  2. I think you absolutely nailed it, 100%. I could not agree more.

  3. Excellent post. I appreciate that you took up the challenge.

    -- James

  4. I would also add that it helps to be able to write children's books. Great bug reports are written with a four-year-old in mind.

    Also, it helps to have people skills. As Brooks pointed out, having a great relationships with the developers means you work as a team and they will be more willing to help you debug their own software as well as point out questionable parts of the their own code. Whereas if you have an adversarial relationship, it's easy to fall into a contest of trying to make the other party seem incompetent.

  5. The post describes main abilities a tester should have. I think, it's useful for interviews preparing.

  6. Great QA engineers dream of bugs in their sleep, and rush to the office the next morning to find that it's a showstopper!