Friday, July 30, 2010

Are You Really a Team?

These days, many of us work in Agile teams or in integrated SCRUM teams. We're no longer the development team and the test team and the design team. Instead, we're one team working together to ship product. At least, that's what we tell people.

If this is you, though, are you really a team? Is this really all one group going toward a single goal?

Three simple questions will help figure this out:
  • Do you assign tasks to individuals?
  • Do developers do test tasks when the team is crunched? Do testers help with system configurations when the team is trying to get set up for new development?
  • When a bug is found in the field, who gets yelled at and tries to figure out how to prevent that kind of problem from making it out to customers again? Your testers? Or your team?
If you're still playing "developer" and "tester" roles, then it doesn't matter whether you call yourself one team or two, you're still restricting yourself. You're still saying that you do your bit, the other guy will do his bit, and if we're all good and lucky, then good things will happen. That's not a team. That's a group of people.

A team is a group of people working toward the same goal. Some have more experience than others in different things, but if you're really a team, then each member will happily do whatever is standing between the team and release.

The difference between a group and a team is very simple:

Teams succeed together and teams fail together.
Groups pass or fail, each individual alone.

If you want to be a team, be a team, not a group.

Wednesday, July 28, 2010

Tales from the Phone Screen

I've been doing a lot of phone screening lately, and there have been some real doozies. I've heard all sorts of things:

"I left my last job because everyone was incompetent but me. It went through two whole teams like this."
Wow - ego coming through! There are definitely incompetent people out there but to run into such a high proportion says that you either have truly terrible luck .... or that there's something wrong with your idea of competence.

"Oh, my weakness is I don't do automation."
This is only strange once you know that the position is for a QA Automation Engineer.

"I'm just looking for a job before my skills get too rusty."
On the one hand, this one is kind of sad. On the other hand, he's not doing anything to keep up or expand his skills, apparently, and that's a problem. I don't want to have to push for every new learning experience.

There are a lot of decent resumes out there. Phone screens really weed a whole lot out, though. Be careful what you say; the hiring manager is probably listening.

Monday, July 26, 2010

Jira and SOAP: Part III

The saga of using Perl code to communicate via Jira's SOAP API. See part I and part II here. In part I, we logged and updated issues. In part II, we were able to get status information about an issue. In our final part, we'll actually reopen the issue:

# Reopen a Jira issue
# @param params{issue} The issue key (e.g., PERMA-12345)
# @param params{assignee} The intended assignee
# @return The jira issue number
sub reopenIssue {
my ($self, %params) = assertMinArgs(1, @_);
$params{issue } ||= '';
my %issue;
$issue{issueKey} = SOAP::Data->type(string => $params{issue});
$issue{assignee} = $params{assignee};
$issue{summary} = $self->_getIssueSummary(issue => $params{issue});
$issue{newStatus} = SOAP::Data->type(string => "3"); #reopened
my $result = $self->_request('progressWorkflowAction', 1, \%issue);
return $result;

It's pretty simple, in the end. We're using the progressWorkflowAction call, which changes an issue from one status to another. We simply create our issue object with all the required attributes (in our case, summary, assignee, the new status, and the issue key) and then call progressWorkflowAction. This call will fail if the user doesn't have permission to make that transition, or if the transition doesn't apply (e.g., attempting to close a closed issue).

And that ends the Jira and SOAP tutorial, at least for now. Good luck!

Thursday, July 22, 2010

Interview Questions: Junior Candidates

When I'm interviewing candidates, I like to ask a lot of questions and present a lot of different types of problems to see how the candidate thinks. There are some things that I look for in more junior candidates specifically. For junior candidates, I'm interviewing for potential more than for what they've done, and that shades what I ask.

"You've been dropped in the middle of the Linux kernel, never having worked on kernels before, and asked to fix a thread deadlock. How do you go about it?"
With this question, I'm looking to see how the candidate learns. Our system is not particularly small, so I want to see how they approach it. Answers that imply they will try to learn the entire system before doing anything are generally not good; this isn't something you can hold entirely in your head without living in it for a while, and I want someone who's going to jump in. Incidentally, we use Linux, so this question isn't totally off base.

"When you're working on a project in a team, how often do you check in? Any tricks or preferences there?"
With this question, I want to see how the candidate works in a team, and how the candidate approaches source control. Some candidates will go away for weeks or months and build a huge, beautiful, complete thing, and then have a heck of a time checking in because all the other code has changed in the meantime (this is a red flag). Other candidates check in near constantly, which means lots of reviews (also a red flag). Every team falls somewhere along this spectrum; I want to see if the candidate matches our rhythm, or is even aware there is a rhythm to match.

"Your team of five has been handed a feature to build: you're going to build the crash recovery mechanism. How do you split up the work?"
Junior candidates won't actually be dividing work, but they'll be on a team, and I want to know how they're going to work with that team. One danger of really junior candidates is that they mostly have experience working on school projects or isolated code areas, and often alone. Our company doesn't work that way, and we need to know that a candidate is going to be comfortable building part of a feature when the other part is being built by a teammate at the same time.

Most of these questions revolve around seeing how a junior candidate works in a team environment, since they likely don't have this much experience. We can teach coding skills, we can teach learning techniques, and we can teach development practices, but only to someone who is aware there is something to learn.

Tuesday, July 20, 2010

Find the Oracle

In testing, an oracle is something we look to that can define the "should" of an application. The oracle (or oracles) tell us what an application should do, how the UI ought to look, how the application ought to perform, etc.

Oracles can take many forms:
  • requirements specifications
  • screen mockups
  • other (competing or complimentary) products
  • style guides (yours or someone else's)
  • similar or related functions in the same application
  • a previous or alternate version of the application being tested (think porting projects)
When you start to test your application, one of your first jobs is to find the oracle(s). This will help guide your future tests, bugs identified, and behavioral expectations.

Sometimes finding the oracle is easy; for example, if you're handed a requirements specification, then you've got at least one of your oracles. Other times, you'll have to get creative.

For example, a friend recently approached me with a problem:

"We're processing files, and we discovered that on this file server with 573million files, we detected 19 million ZIP files. Is that reasonable?"

This is a problem in search of an oracle. It's infeasible at that scale to hand-check all of the files. We're not yet sure we can trust our detection software, though; after all, that's what we're testing. So what's our oracle?

Seems hopeless. "Hopeless" is just another word for "needs creativity". We can do this.

Our problem revolves around the percentage of files on a file server that are of a certain type. There are two ways I think to go about this:
1. run one or more other file type detectors and see how much they agree with my results
2. find other file servers (or reports about other file servers) with a typical file type breakdown. Academic papers, storage vendors, and OS vendors are great sources for this kind of information.

What would you use for an oracle in this situation?

Wednesday, July 14, 2010

Perl SOAP Interface to Jira (part II)

A while ago, we wrote a bit of Perl code that uses Jira's SOAP interface to log bugs, add comments, update tickets, and do other fun stuff. I recently extended this to get the current status of an issue out of Jira. We use this to look for bugs that are closed that we're still updating (whoops!).

Here's the code:
# Issue a request to the Jira server and get a response.
# @param cmd The name of the command being sent
# @param checkResult Whether or not to check the result of the command
# @param params The parameters of that command
# @return The result of the command
sub _request {
my ($self, $cmd, $checkResult, $params) = assertMinArgs(3, @_);
my $soap = SOAP::Lite->proxy(
my $auth = $soap->login($self->{jiraUser}, $self->{jiraPwd});
my $s = SOAP::Serializer->new;
my $doThis;
if ($cmd eq "addComment") {
$doThis = $soap->$cmd($auth->result(), $params->{'issue'},
} elsif ($cmd eq "getComponents") {
$doThis = $soap->$cmd($auth->result(), $params->{'project'});
} elsif ($cmd eq "getStatuses") {
$doThis = $soap->$cmd($auth->result());
} elsif ($cmd eq "createIssue") {
$doThis = $soap->$cmd($auth->result(), $params->{'issueDef'});
} elsif ($cmd eq "updateIssue") {
$doThis = $soap->$cmd($auth->result(), $params->{'issue'},
} elsif ($cmd eq "getIssueById") {
$doThis = $soap->$cmd($auth->result(), $params->{'issueId'});
} elsif ($cmd eq "getIssue") {
$doThis = $soap->$cmd($auth->result(), $params->{'issueKey'});
} elsif ($cmd eq "getStatuses") {
$doThis = $soap->$cmd($auth->result(), $params->{'issueKey'});
} elsif ($cmd eq "getVersions") {
$doThis = $soap->$cmd($auth->result(), $params->{'project'});
} elsif ($cmd eq "getIssuesFromFilter") {
$doThis = $soap->$cmd($auth->result(), $params->{'filterId'});
} else {
croak("Unexpected command: $cmd");
if ($checkResult && defined $doThis->faultcode) { # whoops something went wrong
croak("Error running command: $cmd\nGot: " . $doThis->faultstring);
return $doThis;

# Get issue Status by key
# @param params{issueKey}
# @return The jira issue number
sub getIssueStatus {
my ($self, %params) = assertMinArgs(1, @_);
$params{issueKey} ||= '';
my %issue;
$issue{issueKey} = SOAP::Data->type(string => $params{issueKey});
my $jStatuses = $self->_request('getStatuses',1);
my $jIssue = $self->_request('getIssue', 1, \%issue);
my $jIssueStatusId = $jIssue->result()->{'status'};
my $jIssueCurrStatus;
foreach my $status (@{$jStatuses->result()}) {
if ($status->{'id'} eq $jIssueStatusId) {
$jIssueCurrStatus = $status->{'name'};
return $jIssueCurrStatus;

The first method is what actually submits the SOAP-encoded request to Jira. There are different request types with different parameters, but the ones were interested in are:
getStatuses: takes in the authentication token only, and returns all statuses defined in Jira (regardless of project)
getIssue: given the authentication token and an issue key (the thing you see in the UI: e.g., "QA-1"), this returns the status. Jira returns the ID of the status - for example, "1" - and we use the statuses we grabbed to translate that to a human-readable status name - for example, "open".

Once we have the status we can do things like automatically reopen issues if we're getting logs for an occurrence (hey, it must have happened again, so why not reopen the issue?).

You may choose to do this differently, but hopefully it will give you some pointers on how to handle the Jira interface.

Monday, July 12, 2010

Sprint Renegade

When you're constructing a sprint backlog, there are almost always more things to do than you could possibly handle. Marketing wants some things, sales could close a big deal if only they had some new feature, product management feels like this other thing would be a great differentiator, support has a few requests for additional logging, oh, and development also has a few technical tasks (build restructuring, etc.) they really ought to handle.

So much to do, and unfortunately, those technical tasks often simply don't make it into the sprint. Over time, not having that technical keep up causes increases in technical debt and slows down delivery. Eventually, you have a lot of technical tasks, and yet everyone else still wants new features. Velocity is slowing down, builds are taking longer, and there's more general muttering about things not being clean in the development and test environments.

It's time to introduce the sprint renegade.

The sprint renegade is someone who effectively leaves the team for a sprint. He goes off and simply works on tech tasks to make it easier for the rest of the team to meet the obligations of the backlog. This reduces your potential velocity for that sprint by one person, but it boosts your long-term velocity by taking care of some of the drag on the team. Repeat this for each sprint until velocity is back up where you want it to be.

The general idea of a sprint renegade is that internal tech tasks are very hard to sell onto a backlog, and they will often lose in the face of customer-facing tasks. If that's true in your company, sometimes drastic measures are in order, temporarily.

If your technical debt has crept up to a point where it's interfering with delivering, find yourself a renegade. You'll be glad you did.

Thursday, July 8, 2010

Around the Software

When we're estimating a feature or a story or a software change, we almost always think of the following:
  • time to design the feature
  • time to implement the feature
  • time test the feature
And we come out with an estimate: "2 engineers and 1 tester for a week" (or whatever).

That estimate fails to account for all the other work that needs to be done. We have yet to estimate:
  • test infrastructure extensions
  • other infrastructure extensions (e.g., do we need to provision a hosted site for centralized client logging?)
  • documentation modification or creation
  • configuration analysis and best practices development (to answer the questions customers have)
  • marketing, presentation, and other collateral updates
Just because the product code has been written, and even if the tests have been done, a feature isn't done yet. Don't forget to plan for - and actually do - all the stuff around the software.

Tuesday, July 6, 2010

Code Smell: Model Mismatch

For any system, there are several potential models of that system:
  • the user's mental model of how the system works and how its different parts relate
  • the model in the code, implied by the objects and their relationships
  • the design of the system (hopefully this matches the code, but not always!)
When all the models match, you have an effective system. When the models don't match, that's a code smell. At it's core, a code smell is a warning of future difficulty. And when the user's model doesn't match the code, it's going to be hard to add features the user wants because the code has different ideas.

For example, I'm working on a project that is a basic web application. Users sign up and are put into teams for contests. Everyone on the team that wins the contest gets a prize, and then the whole process starts over with new contests and new prizes. A user made a request to be able to see how much their team had won, total. Sounds fairly simple. It wasn't.

There was a discrepancy in the models. The user's mental model of the system said that "teams win prizes". The underlying system code was that "contests have prizes". This model mismatch meant that instead of simply getting all prizes for a team, the developer had to get all the contests a team had participated in and then get all the prizes awarded for that contest and then filter by team. It was real work!

So the developer tried to refuse to do the request because "the system isn't designed that way." Now we've got a real problem. There are three ways forward:
  • Refuse to fix it. This is tempting sometimes, but it's an unacceptable answer in many cases. It's a user request, and assuming it gets approved, you'll need to implement it. Features aren't chosen by how hard they are, but by their value. Get over it - "it's hard" is not an excuse.
  • Fix it. It's possible to make it work, even if it's in a roundabout way. Note that this does make your code more closely resemble a mountain of spaghetti.
  • Reconcile models. Whether you change the user's model (often implied through the user interface) or change your underlying system design, the right thing to do often is to reconcile the models. This is how you ultimately clean up the code smell - by rearchitecting, refactoring, or modifying the UI and documentation so that what the user understands matches the design and matches the code.
When you get a request or a bug, and your first response is "oh, that's weird" or "oh, that's a lot harder than it sounds", ask yourself what the model is that came to that request. You may have just found a code smell.

Friday, July 2, 2010

Severity What?

I've seen a lot of project orders that include language like this:

"The product shall not ship with any known unresolved severity 1 or severity 2 defects."

Severity 1 is usually defined as crash, data loss, and/or total system unavailability. (In my team, we call this "the big oops".) Severity 2 is usually defined as loss of core functionality of the system.

But when the project's late, those definitions sometimes get changed because, "darnit, we've got to ship!"

And then something like this happens.

Beware changing definitions - there be dragons!