Wednesday, April 30, 2008

Candidates Aren't On a Continuum

We've been interviewing. Still. Some more.

So today we had a candidate in for a second interview. This is our last interview, so by now everyone who needs to talk to the candidate has done so. We sat down together and held our interview postmortem. It was.... okay. Every interviewer had the feeling that the candidate was okay but this wasn't a solid must hire. And then Chip  said something that pretty much exactly reflects my hiring policy:

If it's not a "hell, yeah", then it's a "no".

Either my entire interviewing team is going to think that this is a good candidate who would be a great fit, or else there is a strong risk this would be a bad hire. Given our conservative strategy, any doubt means don't hire. 

It turns out that hiring isn't a continuum - bad candidate, okay candidate, good candidate, great candidate. Hiring is a binary operation - good enough candidate, or not.


Note: I'm well aware that the consequence of this is that I interview a lot of candidates and take a long time to fill a position. It probably also means that I've turned down a great engineer or two. However, it prevents a bad hire and that alone is worth it. I'd rather have an empty slot than drag my current team down by filling that slot inappropriately.

Tuesday, April 29, 2008

Glad It's Your Mistake

The closer you get to release, the more stress there is. You're running out of time to find any issues and prove that all the fixes are there. This is a recipe for making mistakes.

So, particularly when you've found a big bug late, stop, retrace it, and then get some help. You may have made a mistake.

And frankly, that's okay. Better a mistake than a big bug, right? Right.

Monday, April 28, 2008

Say It Explicitly

There is a vast gulf between inferring something and saying something. Saying something brings it in the open and forces consideration of context; other people will actually HEAR you say that thing. Inferring something means you merely think that it's obvious or that "everyone knows that". In reality, your audience could be at a different conclusion entirely, or merely still casting about in search of a conclusion.

This holds most true for the phrase "I don't know". Not knowing is okay, as long as you're explicit about it. If you merely provide context and some details, that's a start, but you're then expecting your audience to infer that you don't know the next thing - the conclusion, the root cause, the reaction. Your audience may very well assume that you know a lot - you've just told them many details - and therefore that you know the next thing.... but you're holding out on them! From this point things deteriorate quickly. So head it off; say "I don't know".


A (For Once Non-Software) Example:
Over the past weekend, my apartment building lost gas service. No hot water, no heat, etc. This started early Saturday morning. Sunday afternoon a note was slipped under everyone's door. The note explained in great detail that a water main a block away had broken and had leaked into the gas line. So the gas line had to be emptied, dried out, repaired, and checked. After that work was all done, the building gas lines would have to be checked and the system brought back up.

So what was not said?

"I don't know when we will have hot water again."

I should note that building management is more reactionary than proactive about communicating with the residents in general. I should also note that the morning newspaper had an article about the water main break (it was a big one!) and indicated in that article that gas service was likely to be restored Sunday. Because building management didn't say "I don't know", residents were left to assume that hot water would be restored sometime Sunday and that building management was simply not being communicative. 

There were a lot of angry residents come Monday morning, and all because "I don't know" wasn't said!

Friday, April 25, 2008

It's The Same, But...

I just got a new computer at work (hooray!). Along with this shiny new computer came a shiny new operating system. That's right, ladies and gentlemen, instead of running Debian Sarge I'm now running Debian Etch.* All is good, except one thing: Firefox is gone.

This is a big deal. We have a lot of customers who work in datacenters, a fair number on Linux boxes, and supporting Firefox on Linux is one of our requirements. So where's my Firefox?

Turns out... my Firefox has become Iceweasel (it's opposite day! get it?). A quick search turns up some of the history here, and it ain't pretty. Basically, Firefox is not "free" (as in software) enough for the Debian folks, and the two groups can't reconcile, so Debian took the free enough parts of Firefox, rebranded the thing, and put it in their distribution.**

Now, I'm not going to step on anyone's zealotry, but I really need my core Firefox back.

Why isn't Iceweasel good enough?
  • It's already not the same as Firefox. There are some "additional security patches" in Iceweasel.
  • It's several versions behind Firefox, at least as of this writing. Looks like 2.0.0.14 for Firefox and 2.0.0.9 for Iceweasel.
  • It's a fork. While the browsers may be nearly identical now, there is no guarantee that they will remain so. I'd hate to look up in six months and realize that I wasn't testing what my users were going to see.
So how to get real Firefox?
This one took a bit of digging. I basically wound up following the Debian User Forums post, with a few tweaks. So, here's what I did differently:
  • Keep Iceweasel. The post assumes that Iceweasel is uninstalled. Instead of removing it, I just removed the /usr/bin/firefox -> ../lib/iceweasel symlink. Then I did ln -s /usr/bin/iceweasel /usr/lib/iceweasel to make a symlink for the new browser. Now when I want to run Iceweasel, I just have to type "iceweasel" in my terminal.
  • Not Run As Root. I find it easier to just sudo, but this one is totally personal preference.
  • Not Install libstdc++5. I already happened to have this on my clean Etch install, so I skipped this step.
  • Change the Iceweasel menu item. After I finished the steps, I just changed the iceweasel menu item to open firefox instead.
Everything else, I did by the book.

Moral of the Story
There are two morals to this story. First, you can run standard Firefox on Debian Etch. Second, and more important, if something is the same except for a few changes, it is absolutely not the same. Don't be different just for the sake of convenience; remember your users and use their tools.



* On this machine, anyway. I've written before about the myriad operating systems I encounter in a given day.

** A sidways rant in two parts:
Part 1: Iceweasel isn't an inherently dorky name, but in context it becomes truly stupid. I like the logo, though. I'm more of a blue person than a red person in general, and the weasel is cute.
Part 2: This in my opinion is part of what makes people afraid of and confused by open source software. Debating degrees of freedom is incredibly opaque to your average user. Now some poor sucker experimenting with open source has to google past flame wars and "the truth about XX" "no no the REAL truth about XX" to get any real information. I know people get all up in arms, but this is really just petty and makes us all look dumb.

Thursday, April 24, 2008

Unintended Tests

We're in the middle of testing for a release. Machines are being installed, brought up, taken down (nefariously or cleanly, depending on the test). It's all going gangbusters.

And then yesterday at about 3:10pm, the power went out.

Thanks, NSTAR, for the tests we hadn't intended to do at precisely that moment!

Wednesday, April 23, 2008

Can You Say Yes or No?

We're nearing a release, and that always brings up the interesting questions:
- Is It Ready?
- Should We Ship This?
- Okay, QA, Make the Call

All fine questions. All questions that QA alone cannot answer. So, where does QA's power/job/role start and where does it end? In my head, it's pretty simple. QA can stop a release, but QA cannot allow a release to go forward.

QA can:
- stop a release
- change a release from general shipping to limited ship. This is a milder form of holding a release.
- describe the current state of the release, including what is known and what is not
- characterize the risks and concerns in releasing at any point

QA cannot:
- decide to ship a release.
- work in a vacuum. Business input is needed to help weigh bugs.

I'm aware that this is somewhat controversial; it's not uncommon for a management type to say that the release has to ship over QA's objections. Usually the rationale behind this is a deal or revenue: "We have to ship in Q2 or we can't close this deal!" The short answer to this is that you really should have a QA lead that you can say these things to, and the QA lead can consider that in holding a release (or altering it to meet the terms needed while limiting exposure).

Tuesday, April 22, 2008

Phone Screen 101

I would have thought this was obvious, but maybe not....

If you're doing a phone screen, don't eat!

That is all.

Monday, April 21, 2008

Feeling Patriotic

Sorry no post today... it's one of those wacky state holidays!

This one is Patriots' Day.

Enjoy the spring weather, everyone!

Friday, April 18, 2008

Follow the State Transitions

One of the types of testing I like to do is state transition tests. This is useful in linear or near-linear workflows. Think the types of things that lend themselves to a workflow diagram.

Why?

The errors are in the gaps between the boxes.

Creating Your Test Cases
Ultimately, state transition tests are a tree search in which you must hit every node. I tend to use a breadth-first search, but you can do a depth-first search. The trick here is that you're not looking for one result, but for all possible paths.

An Example
This is a very very simple example:
Screen 1: Type in the IP address desired. OK / Cancel
Screen 2: Are you sure? Yes / No

Transitions from state 1 (format: input -> action -> result):
  • Valid IP -> click OK -> Screen 2
  • Invalid IP -> click OK -> Return to screen 1 with error message
  • Valid IP -> click Cancel -> Shut down machine
  • Valid IP -> turn off machine -> Machine shuts down
  • Invalid IP -> click Cancel -> Shut down machine
  • Invalid IP -> turn off machine -> Machine shuts down
Gotchas
The biggest gotcha with this method is that it creates a LOT of test cases. This is a case where you're going want to go through after creating your test cases and prune, prioritize, and/or automate.

Give it a try, and good luck!





Thursday, April 17, 2008

Work In Progress

I received a test plan yesterday from someone who said it was a work in progress but wanted me to do an estimate on how long it would take to actually perform the tests in the test plan.

Awesome, no problem. So I open up the test plan, and I see the following:

------------------------------------------------
Test Strategy
We will test the product () thoroughly and completely. We will perform tests of the product under production-like conditions.

Success Criteria
There will be 0 defects identified in the field over the life of the product that were not found prior to shipping the product.

Tests: Feature
TBD

Tests: Other Feature
TBD

------------------------------------------------------

I sent it back and said that I couldn't estimate it before there had been a little more progress on this work in progress!

Wednesday, April 16, 2008

Working Definitions of Failure

There are several failure terms that get thrown around rather loosely. That is more than a little imprecise. All of these terms have general definitions, but let's be even more precise than that; there will be a working definition that gains precision for your environment.

So, let's define some failure types:
  • Abort. Abort means some process caused your program to stop. There was no failure state, but there was no lack of failure state. The program (or process) simply stopped. For example: on Windows, IOMeter will abort if the underlying mapped drive goes away; it doesn't hang, it doesn't shut down. It simply stops the test.
  • Fail. Failing means that something was asserted and got an unexpected value. This is trying and failing. Failure is an active state.
  • Hang. Hang means that the program simply stops doing anything; no perceivable action is taken. This one is very dependent on your application - loading a web page taking half an hour is probably a hang; copying a 1 TB file taking half an hour is definitely not a hang. At my current place, we almost never say hang. We say "nonresponsive for time X" or "has been performing action A for time X, which is unexpectedly long". This is mostly because we have a lot of long-running operations. 
  • Takes a Long Time To. Takes a long time to means that you expected some operation to take duration X and it took significantly longer. Precision is better here - took 20% longer than expected, etc. Sometimes you can't have precision, though; in that case, allow at least 25% over the expected duration before you start saying that something is taking a long time. If it's under 25% longer than you expect, say "longer than expected", which sounds less judgemental!
  • Crash. Crash means that the program you're running stops and is no longer running. If you do a process listing (ps aux or whatever), you don't see the program any more after a crash. Don't say crash unless you mean crash; that's a word that is likely to cause a bit of panic.
  • Error. Error means that something didn't happen as expected, or happened and was not expected. An error is not necessarily a failure; it's not specifically tied to an assertion. An example of an error is a test that dies in the setup method.
In the end, all of these are just words. There are two considerations when using these words: precision, and consideration. Use these words precisely so no one goes and chases a red herring. Use these words carefully because flinging them around only makes you look judgemental.


Tuesday, April 15, 2008

Caring Vs Competent

There are a couple of interview adages that I've come to believe:

Adage 1: Many many people are simply not competent.
Adage 2: Never mistake interest for competence.

One of the things I've noticed is that there is often an inverse proportion between competence and explicit preparedness. This gets more likely the more senior the candidate I'm interviewing. Let's define some terms here:

  • Competence: The ability to do the job we require. This expects a facility with common tasks and underlying principles to the level required by the position. With a more senior position, competence indicates the possession of good habits and skills - scripting ability, test design techniques and skills, a bug-finding toolkit. With more junior people, competence is about ability to learn, underlying talent and having successfully avoided bad habits.
  • Explicit Preparedness: This is the thing that a lot of job interview sites will tell job seekers to do. Find out about the company and ask about market conditions, competitors, how they can help with market trends and recent company announcements.
I've been interviewing for senior-level QA engineers lately. The really competent ones have known some about the company but not a lot; I suspect they've spent about 5 minutes on the website. The ones who have lots of questions about the company and want to talk about what they can bring to the table based on our latest patent, etc...... have totally bombed the competence area.

I suspect they know their technical and testing skills are a bit weak and are trying to look "well rounded". Well-rounded is good, but that means the candidate should have both. Good candidates, ultimately, are about balance. Show me you care, but show me you can do the job. 



* You can always tell when I'm interviewing because this blog fills right up with interview posts!

Monday, April 14, 2008

Magic Numbers

One of the more common test techniques is boundary value analysis. Basically, you find the edges of allowable input and test each of the boundaries and also off by one values. For example, if you were testing a text field that allowed you to enter valid months, you would test:
  • 0 (invalid but off by 1)
  • 1 (lowest allowable value)
  • 12 (highest allowable value)
  • 13 (one above the highest allowable value)
That's all well and good, and there are lots of techniques for determining your boundaries when you have various inputs.

Defining Magic Numbers
I assert, however, that inside your application are other boundaries, often hidden from the user. These are what I think of as the magic numbers in your application. Some of these magic numbers are imposed by external forces (time is a big one, for example). Others are contained within your application. Magic numbers are the values around which your application is sensitive.

Finding Magic Numbers
Finding some magic numbers can be very easy, but others are deeply hidden. To find magic numbers, consider the ways your application can be sensitive. Common areas of sensitivity are:
  • Time. Background processes - cleanup, auditing, notifications, etc - are huge culprits here. Think of things that happen every n hours, minutes, days.
  • Size. This is usually a scalability control in place. Common examples are file-system or database areas - log rolling, indexes on tables over size n, creating a new directory on the file system when an older directory has more than n files in it.
  • Frequency. This one is a little less obvious, and is sometimes lower-level than most of your application - garbage collection, background processes triggered every 5 messages your system handles, etc.
Universal Magic Numbers
Some magic numbers are outside your application and apply to almost anything. These are units of time. Your application probably won't be sensitive to all of these, but it will very likely be sensitive to some of them:
  • Leap Years. February 29th is a common breaking point.
  • Daylight Savings Time. Switching on or off daylight savings time is a common place where things break; skipped or repeated jobs are likely.
  • Time Zones. Look for something to be off by your distance from GMT (or UTC). I'm GMT-4, currently, so I look for times that are 4 hours in the future.
  • Max Directory Sizes. This will vary by OS, but try looking to see if we're using a file system directory generator that rolls on you.
  • Max File Sizes. This will vary by OS also, but look to see if the OS is rolling files on you.
  • Min File Sizes. This one is a lot more rare, but watch that you don't get issues with an empty file. Occasionally you'll find they handle differently.
  • Every n Ticks. This one is really rare, but it's worth looking at. Check for things that happen every n clock cycles. When the CPU gets really busy these may start to vary a little bit.

Application-Specific Magic Numbers
In addition to the external forces acting on your application, there are likely internal magic numbers. These are the heartbeats of your application, and if you find them, you can stress them. These will vary by application, but look for some of these:
  • Log rolling. Be sure to check this as you're changing log levels.
  • Background processes. Cleanup, notification, and auditing are common ones.
  • Heartbeats.
  • Areas that interact with the file system. Message processing, FTP handling, etc.
  • Timeouts. Don't forget to include the timeouts of other programs you might be using - SSH, etc.
Now What?
Once you've found your magic numbers, you can apply boundary value analysis techniques to stress them. That's not always easy, but it's a good source of some really nasty and subtle bugs.

Happy hunting!

Friday, April 11, 2008

Eleven Million Copies of the Same Error

We have a great test infrastructure at work. We run tests automatically every night, and they do this*:
  • start the test run
  • do a build
  • pick which test suites to run
  • reserve machines for the test (from our lab - with 300+ machines, you want a reservation system!)
  • run the test suite
  • gather logs from each machine used in the test
  • parse the logs to figure out whether the test failed
  • updates a list of failing tests and count of tests run
This is great, and generally works quite well. We come in every morning and have access to a central summary of exactly what failed and why, with all the logs right there. The lab just hums along in the background, doing other things.

There's one issue here, and it's really a general QA issue. Sometimes we come in and we've had a lot of tests in a lot of suites fail.... and all the errors look the same. One common example of this is ssh failures. Sometimes we'll come in and see 100+ failures all with the error "could not ssh to . No route to host."

So. Is this one bug, or many bugs?

Arguments in favor of one bug:
  • No one likes duplicate effort. That goes for bugs, too.
  • If it's the same problem, it's almost certainly the same fix. One problem, one fix, one bug (has a nice ring to it, huh?).
  • Don't get more people that necessary looking at the same bug.
  • Helps recognize the extent of the problem.
Arguments in favor of many bugs:
  • Bugs are a lot easier to merge than to separate.
  • There is a risk you're lumping multiple problems together under one issue.
In general, I will log many bugs and reference them ("looks like bug 42321") from each bug. That way it's easy to merge later. I tend to be risk averse, so I'd rather have more work and lower risk than less work and higher risk. Once we get the signature of the problem (how to differentiate it from similar problems), then we can feel free to consolidate.

How do you handle similar errors?


* For those of you who work with me, yes I'm simplifying a bit.

Thursday, April 10, 2008

Great Comments

Sometimes the comments really make the code reading worth it. I just found this comment describing the purpose of a method in some source code:

"Repeatedly polls the future until it is done"


I think that might be the best thing I've read all day! And yes, in context the comment makes a lot more sense.

Wednesday, April 9, 2008

Slippery, Buggy Slopes

As you settle into a routine, it's easy to start cutting corners. When logging bugs at a fairly rapid clip, this becomes even more likely. Let's face it, actually writing up bugs isn't the most interesting part of the job. However, it doesn't take long for bugs to start losing important context. Then you wind up with a bug like this:

I downed a switch and the GUI hung. Waiting 20 minutes caused it to start responding again, but extremely slowly. Logs attached.


Ick ick ick. We've already talked about poorly-written bugs, and this is a good example of one. So why does this happen?
  • Writing up bugs is boring. Freely admitted.
  • You know the context, so you don't write it all down.
  • When you're logging several bugs in one area, it's easy to cross reference them rather than write each bug up fully.
But... understanding why doesn't excuse it. That's still a slippery slope to go down. The obvious reaction is to state that all bugs must contain certain things:
  • setup information (the state you were in when you started reproducing the issue)
  • steps to reproduce the issue, including data you used
  • logs if the situation is at all time-based or involves multiple steps (e.g., misspellings don't need logs)
  • screenshots if there is a GUI
  • version of the software in which the bug occurs.
I think this is a pretty good basic list and not overly onerous.

So, having found a good goal, what can we do to help ourselves reach that goal consistently? I've found two big things that help. First, dev should kick back any bug to QA that doesn't have this information. If the person who found the bug is available, there's no reason to start an archeological expedition to figure out what's going on. Second, make getting the defect information very very easy. I've found some tools helpful:
  • a log gathering tool is invaluable, particularly in a multi-machine system
  • have a fast defect tracking system. Don't put it on some old hardware that will make users wait.
  • good easy screenshot tools are helpful. SnagIt is better than printscreen.
  • If the situation is complicated, a recording/playback tool is useful. They make these for GUIs, and you can also make one for your own system.
Long and short, logging bugs isn't the most fun part of QA, so make it easy to do it right. Then, keep doing it right. Use your team and use your devs and keep yourself taking all the steps.

Tuesday, April 8, 2008

Interviews Are Tests

Yesterday I had a candidate in for an interview. Now, one of the things we do in interviews that's kind of odd is a fair amount of semi-formal testing. It's nothing out of the ordinary, but we perform a couple of exercises to see how candidates approach various testing problems. For example, we hand over a spec and ask the candidate to design a test strategy for it. Or we ask candidates to sit down in front of a program and find bugs.

For the record, there are no "how long does it take to move Mt. Fuji ten feet to the left" questions.

Every once in a while I get a curve ball. A candidate will come in, I'll explain what's going to happen today, and he'll say, "Oh, I don't take tests."

I hate to break it to you, buddy,* but:

Interviews ARE tests.

The interview is a chance for the company to test the candidate. It's a way to assess skills, to see how the candidate reacts under stress (even if you don't need the job, interviews are stressful). It's a way to determine whether the candidate fits in with the company culture, and to figure out if that glowing resume is real gold or fools gold. Without actually testing the candidates knowledge and claims, it's very difficult to tell the difference between a good candidate and a good resume.

This:

Is not the same as this:**


On the flip side, the candidate gets to test the company, too. How does the manager react to bad news? Is there any sort of scary culture aspect? Does the company support the candidate learning and furthering his own career?

Every time I interview a candidate, I know I'm taking a test. And the candidate is taking a test, too. So as far as I'm concerned, any candidate who "doesn't take tests" is not really interested in interviewing.




* Can you tell I really don't like the attitude? And actually, I don't really hate to break it to you. It's just an expression.

** For the record, the top picture is fool's gold (aka iron pyrite) and the bottom picture is real gold.

Monday, April 7, 2008

Code Coverage Complexity

I'm working on a project that is looking at using rcov for code coverage. Makes sense; it's a Ruby on Rails project, so rcov is easy to integrate and even ties nicely into the build system.
So what kind of code coverage do we want? Easy! 100%! Every line of code must be tested!
Err.... let's talk about that one for a minute.

Different Levels of Coverage
There are several different types of code coverage: C0, C1, and C2.
C0 code coverage tells you whether a given line of code was executed.
C1 coverage tells you whether each branch of each line of code was executed.
C2 coverage tells you whether each code path (across lines) was executed.
For example:
foo ? bar : baz
Let's say I have just one test (pretending for the moment that this is at all complete):
def test_foo
    foo = true
end
What's my coverage?
C0: 100% - I executed that line
C1: 50% - I executed only one branch of that line - the true path.
C2: unknown, but no higher than 50% (it depends on the rest of the program, which isn't shown here)
Different Types of Code
Coverage tools include and don't include various types of code. Depending on the tool you use, it might include third-party libraries, or not include view code, or not include javascript.
Coverage Goals
So with all of that, what's a reasonable coverage goal?

Well, that will ultimately depend on the tools you pick. So, in code that you own, and in code that you can measure, aim for C0 coverage of 90% or better. Go for it. The more complex the coverage (C1 or C2), the lower the coverage level you can expect.

Friday, April 4, 2008

Heard in the Phone Screen

Put this in the category of "candidates say the darndest things". 

Candidate: "I've been doing performance tests; I haven't actually gotten to write tests in  a long time and I really want to. I want to look at emerging technologies and get into writing tests and test harnesses, not just running tests."
Me: "So what have you been doing to prepare yourself for such  large transition?"
Candidate: "Um.... well... I write code every day, just not writing tests. I think I'm already prepared."


From the same candidate:
Me: "How would you characterize your expertise at Perl?"
Candidate: "Oh, I'm very good at it. You can ask me anything."
Me: "Okay, tell me what the major difference is for you between object-oriented Perl and standard Perl."
Candidate: "Oh! Object-oriented perl is just Python. It's really just a change in how you set up in your GUI environment."


Confidence is great. I'm a huge fan of confidence. But if you're going to be confident, please be correct!

Thursday, April 3, 2008

The Daily Report

I've noticed a trend recently. A couple of projects I work on or am aware of are feeling a need for project status reporting. One of these is in acceptance testing for a release. Another is just getting to the end of the first dev cycle and marketing/product management/other dev teams are just getting their first look at it.

What's In  A Report
In both of these cases (and in many other situations), there is a common set of information in the report:
  • an idea of what's done and what's not yet attempted
  • an idea of what issues there are and how bad they are
  • where to go for more information
Constructing the Report
Here's where we start to run into trouble. The contents of the report aren't usually in one system. Ideally we'd like to just run a query (in our defect tracking system, our ticketing system, our project management system, whatever. Often you're looking at multiple systems, however. If you're lucky, it's just two. If you're not, it's more.

The tempting thing to do is say to anyone who wants a status report, "Sure! Just go look it up whenever you want it!" Let the person who wants to know go figure it out. Be nice and send over the correct queries (or links that will do the queries for them).

Push vs Pull
Where our grand plan of "go look yourself" falls down is that reporting generally works best when it's push, not pull. It's a bit harder on the generator, but easier on the recipient. And the king of push is..... email.

So often we wind up constructing the report and emailing it out every day. This also provides the ability for the report generator (the person) to add a little commentary. The best way I've found to do this is to use a script to construct the basic report, and then to send it out by hand.

Downsides
The problem is that any time you do reporting outside the system that actually holds the data, you:
  • Create work for yourself (whether writing a script or hand munging data)
  • Disassociate people from the location of reliable information. They can't get more info from email, and you're training them to look at the email, not the systems.
  • Drop information. You don't put everything from each system into your report, so some nuance is lost.
  • Duplicate information. This is bad in the same way code duplication is bad.
  • Remove the sense of time. Each report stands on its own, and getting the history of any issue still requires you to go to the primary source.

All that, and yet I still email out every night, and this group I'm watching is doing the same thing. Anyone else have a good idea how to balance push vs pull with aggregating data from multiple systems?

Wednesday, April 2, 2008

Wolves Are Pack Animals

Well, it's about that time: I'm hiring (yes, again).

So we've geared up, and done all the job startup stuff:
  • decided what we're looking for
  • written it up in a (hopefully compelling) job req
  • figured out a preferred salary range
  • called the recruiters who have introduced us to good candidates in the past
We've gotten a few resumes in, and I've done a few phone screens. Twice now I've had an otherwise good candidate say something like: "I'm a lone wolf."

Okay, back up. Is this a good thing to say? A lone wolf is typically someone who prefers to (or can only) work in isolation.

Any candidate who says that he/she is a lone wolf is very unlikely to be hired. Why?
  • We're an XP shop. Lone wolves and pair programming don't go well together.
  • A lone wolf has a connotation of being unwilling or unable to work with others. We're a QA team and one of the reasons we work as well as we do is that any of us can ask anyone else a question and not look dumb. (It's amazing how much that alone increases velocity!)
  • Most of the people who believe they work best in isolation have produced code that is very difficult for others to use. Most of the code QA writes is used by others. These two do not go together well.
So I'm not sure where the pride comes from. Independent, self-motivating, able to take on a large project and define it - all those are great things. But lone wolf?

Around here, wolves run in a pack.


Tuesday, April 1, 2008

Story Containers

We follow the XP development process.*

One of the things that we're having a bit of a time with is reconciling two goals:
  • stories should be of manageable (small) size
  • stories should have clear user benefit
Sometimes this is easy. 

For example, a story like this:
"Customer can email support from within the support page in the application"
has clear customer benefit and is pretty small.

Sometimes this is not so easy. 

For example, a story like this:
"System automatically logs a bug in the event of a crash (or updates an existing bug)."

This story is quite large, so ideally we'd break it down into smaller stories. However, doing that puts us at risk of losing the customer focus. One of the smaller stories in our example might be "System can log in to the defect tracking system." Fun, sure, but not directly helpful to the customer.

The best way I know of to handle this is to have multiple stories that are grouped into a story container of some type. This way we can keep our stories small but still say that they are helping provide benefit to the customer. The problem is then differentiating story containers from stories in the queue. I've tended to keep them separate, but this is far from a solved problem.

Anyone in the ether have ideas?




* Mostly, sort of, etc. Don't email me with process rants, please!