Some ways to measure this are really good. Typically these are a complete pain to gather, but if you're really interested in measuring this on a quantifiable rather than a gut level, you've got to start digging hard.
- On Time Ratio. Measure how on time the person is with tasks and projects. The idea is to over time average how far off the person is. For example, someone who completes a 10-day project in 9 days has an on-time ratio of -10% for that project. Someone who does the same in 11 days has a +10% on-time ratio. The idea is that over time, for productive and effective employees this should approach 0. That is, the person should, over a number of projects, prove to be accurate. Note that someone with a high negative number (so someone who is generally finishing early) is not necessarily effective; they're just padding estimates.
- Management Ratio. I run a very self-directed team; my title is Lead, not Manager and there's a reason for that. I don't spend all of my time managing someone. Sure, things come up, but it shouldn't be a constant. The management ratio defines how much time you spend with each person performing management duties. This is a comparative matrix. If you're spending 0 time with a person, then something is wrong and you the lead probably aren't helping him be as effective as he could be. However, spending too much time with someone is a drain on the whole team. So, measure how much time you spend on management activities with each member of your team. If some team members are taking inordinate amounts of time, then there's a problem. The specific definition of "inordinate" will vary a little bit, but anything over twice your average is probably bad.
The good thing about these metrics is that gaming them is exactly what you want. You want your tester to try to game the system by making estimates really accurate. You want the person working for you to game the system by not requiring a lot of hands-on management.
Bad measurements tend to be really easy to gather, and they look like they're telling people a lot. In reality, they are incredibly easy to game without actually helping the team.
- Number of bugs found
- Number of bugs verified
- Number of test cases written
- Code coverage
- Run length of automated tests
All of these items fall short because improving or increasing them does not have an inherently positive effect on the employee, the team, or the system under test. Interestingly, these measurements tend to be automatically gathered and tend to masquerade under the fancy-sounding term "metrics".
Some metrics treat employees like real cogs in a wheel. I hope by now everyone has given up on these:
- How many hours you work.
- How many modules you have tested.
- How much you talk in team meetings.
This is both dumb and short-sighted. Just because I can see someone doesn't make him effective.
Ultimately, a good measurement of an effective employee gets back to gaming the system. Accept that whatever system you use will be gamed. The tricky part, then is to set up the system such that gaming it produces desirable results.
What other measurements or incentives do you use?