Powerful ProductionBy John Parkinson | Posted 04-01-2003
Strong Signals: Star Search
Back in the late seventies and early eighties, when I managed large, complex software projects for a living, I kept track of which programmers were contributing the most to the code we delivered to our customers. A typical development team consisted of eight to ten programmers writing and testing code for three months or so. We usually had about a dozen teams contributing to a development release, so each year I accumulated 40 to 50 sets of productivity data on both teams and individual programmers, covering the efforts of about 100 people. Between 1978 and 1983, I built up a pretty decent performance database.
The results were hugely skewed. Although the productivity of the teams didn't vary much (and everyone worked about as hard as everyone else), almost 75 percent of the final code came from just 5 percent of the programmers. And just about 100 percent of the "best" code (few initial defects, excellent efficiency) came from this same small group.
This was my introduction to "Power Programmers"sometimes called 10x programmers because they are an order of magnitude better than everyone else. Power Programmers achieve this performance without turning to sophisticated tools or methodologies and without necessarily requiring heroic work effortsalthough they do tend to have odd or very individualistic working habits. I could never find very many of themin my best year I had around ten out of a staff of nearly 150and they were pretty hard to keep around for the long term. Yet it's amazing to look back at how many great software products (think Lotus 1-2-3) and systems (think everything Internet pre-Netscape) we owe to these supercapable programmers.
As I got more involved in large-scale software projects and developer productivity, I almost lost sight of the Power Programmer group. Focusing on tools and methodologies aimed at improving the performance of the average programmer seemed like a better investment than chasing these rare and fickle individuals. The advent of highly configurable software packages also seemed to run counter to Power Programmer processes.
There were still some corners where Power Programmers could feel at home. Many found their way into the open source movement and the development of Linux. And some parts of the Internet craziness (the parts that actually worked) were havens for some amazingly talented people. Then the bubble burst, and we got back to a life of improving the productivity of the average programmer.
Just this year, I've come across a couple of software products that suggest Power Programmers are coming back into the mainstream. The first is Squeak, a sort of grown-up version of Smalltalk, which a few people will remember as the first real object-oriented development environment. Squeak encompasses an operating system, interactive GUI, development environment and tools that can run on anything (including a chip without a built-in operating system) with minimal porting effort (think a couple of weeks by a couple of people). The Squeak kernel code is only a few hundred "objects," which is why it is so easy to port. Squeak was developed entirely by Power Programmers, led by Alan Kay. It's an amazing example of what Power Programming can achieve. Take a look at www.squeak.org.(Squeak is open source too, and it will be interesting to see how, or if, it survives broader exposure.)
The second product is an IP Multicast collaboration and videoconferencing system called inSORS Grid (see www.insors.com). Based on research from Argonne National Laboratory, inSORS supports multisite collaboration over the broadband Internet without the complexity of traditional teleconferencing systems. With enough bandwidth and display space, dozens of sites and hundreds of people can participate without losing track of what's going on or who's driving.
I'm using these products simply as examples, without implied endorsements. Since discovering Squeak and inSORS, I've found quite a few other Power Programmer groups and individuals writing great code very productively. Data compressors, video codecs, security products, speech recognition technology, even developer tools are attracting Power Programmers. Few of them use the tools and methods of corporate IT, or even of large software companiesand they don't seem to be missing them a bit. Nor do they pay much attention to the codified processes and principles of software quality management, whilst still producing stable, efficient code fast and efficiently.
With developer productivity and development process effectiveness high on a lot of CIO agendas these days, I'm starting to think again about applying the practices of Power Programmers more widely. I started by going back to my original data from the Seventies and Eighties. If just a few people were writing most of the code, what did the others do? Maybe if I could just identify the right people, I could get by without the rest.
As you might imagine, things were not that simple. Very few of the people on my teams contributed nothing at all, but a lot of them didn't contribute very much. I would have gotten a few additional points of productivity by eliminating a few people who were, in effect, "defect generators"just no good at writing code.
I also discovered that although experience mattered, several of the Power Programmers were actually quite junior. And there was some work Power Programmers just wouldn't dotesting, documentation, release management and so onthat needs to get done if projects are to be commercially successful. The less accomplished programmers often ended up with in these roles.
Finally, I discovered that the results of Power Programmers correlated strongly with the ability of the project managers coordinating them. My best managers knew how to get the most out of their Power Programmers, without overtaxing the less-talented people. Project managers who went strictly by the book never got the same level of performanceand usually lost their Power Programmers pretty fast, either to other teams or to other employers. This gave my release managers a hard time. They never really knew where their best teams were on the schedule, yet these teams almost never missed a deadline. The teams that did report precisely where they were on the schedule were often late and needed frequent resource adjustments (although we never missed a release date in five years).
What can we learn from this? I'm not yet ready to suggest that we can replace the whole software development process with Power Programmers. But I am just about ready to suggest that we have gone too far in trying to normalize performance in software development, and that we may be following a blind alley as far as future capability models go. We may have simultaneously made software development so restrictive, and yet so complex, that programmers good enough to do what we need can't tolerate the methods and measures we make them use.
In some areas of software development, automation will be the right answer. But automation has its limits, and examining flexible manufacturing analogies to see where we can leverage the very real capabilities represented by Power Programmers looks like a really good idea right now.
It won't be easy. We will have to change a lot of things in corporate IT to attract and (to the extent possible) retain Power Programmers. And we'll probably have to upgrade significantly our management and measurement processes. But there are some things you just can't pull off with brute force, some problems you can't solve with average people, no matter how well trained and equipped. For these problems you need the most talented people you can find. You need to find a way to incorporate some Power Programmers.
Then sit back and watch in admiration. The results may amaze you.
John Parkinson is chief technologist for the Americas at Cap Gemini Ernst & Young. Please send comments on this story to firstname.lastname@example.org.