Ultimately, we as engineering teams generally have a few fairly concrete goals:
- deliver functionality to our customers as fast as possible
- don't kill ourselves - we make mistakes when we work too fast
- create solid software so that it can survive the onslaught of usage (yay success!)
- provide management with predictable production rates - they need to know when things are likely to happen, even if it's just an estimate. Most of the time we call this velocity.
None of that says anything about specialist or generalist, at least on the surface. If we have specialists, some parts of our goal become easier; we are more likely to create solid software, and we'll probably do it a bit faster. If we have generalists, though, we're more likely to be predictable; one person being gone doesn't stop everyone else from finishing things, and we have fewer bottlenecks since anyone can jump in to help.
Specialization is occasionally necessary. If you're doing something truly specialized - a new data storage technique, or high volume (read: millions per second) data processing - then you need a specialist. Or two or three. For most of us, though, we're not doing anything that specialized. We're building web applications, or apps for various devices, or installable software. We have to be good software engineers, but the problems aren't really that esoteric. Even in cases where the product is hugely specialized, there is still a large part of it that's just a solid program to do the more commonly needed parts (administration, logging, UI, etc.).
It's a lot like physicians. Most of the time what you need is a general practitioner who is going to know how to solve lots of different kinds of problems that many people need solved (chicken pox, colds, appendicitis). You really only need a specialist when something truly weird is going on - Wilson's Disease, for example.
Here's the rant part:
There is another, more insidious, form of specialization that I like to call "learned specialization". This is where a person or a team who could be a generalist decides to become a specialist. This is what's happening when you hear statements like, "I'm the architect. Therefore I have to do all of the design and you can't start that feature yet because I haven't designed it." Or you hear things like, "Well, Jeff should do that because he knows the frobble module." Or "I'd really rather not tackle that story because I haven't worked in the admin before." The team - or one person on the team - is creating unnecessary specialization. There's no reason that John can't learn the frobble module, even if Jeff already knows it. There's no reason that Jorge can't produce a design for a feature and get the architect to take a look (must assuage that ego!).
The worst part of learned specialization is that it's self-reinforcing. The longer that Jeff works on the frobble module, the better he'll know it and the more other team members will just avoid those tasks because Jeff is slowly getting faster and better at working with it. The more Janet avoids the admin module because she doesn't know it, the larger it will get and the more there will be that she doesn't know. If the architect does all of the design, no other team member is going to learn how to design software effectively.
Be vigilant against learned specialization. Any time you see incipient specialization occurring, take it as a training opportunity and get someone else in there. Ultimately, long term software success is about breaking down silos, creating an environment in which there are multiple points of redundancy, and encouraging an atmosphere of training rather than avoidance and burnout. You'll only get there with generalization.