CIO Insight: It's depressing that some things never seem to change. Your discussion of Frederick Brooks' 1975 book, The Mythical Man-Month, explains how adding programmers can actually slow down a project. Brooks wrote the book about a huge IBM project gone bad in the 1960s, and yet it's still relevant today.

It is an amazing book. I wish every developer and manager would just sit down and read the first few chapters, and then get back to work. Brooks talks about the original software disaster, with the biggest company of its time betting on a system that would change everything, and the entire software component of the project was a year late. Now, the flip side is that the IBM 360 did become the foundation of corporate computing for the next quarter-century, so the delay in creating its operating system didn't ultimately sink the project, or IBM. But it was an early indication that this whole undertaking of writing large software projects was not a simple matter. And Brooks, an extremely thoughtful man, made it into a great read.

CIO Insight: So where's the improvement over time? Are we just being impatient with a branch of knowledge that is still fairly new? Or is there something inherent to software development that makes it so weird and vexing?

There is an optimistic side to this. I tried to include it in the book by interviewing smart, articulate people with different points of view. You get one perspective that says, hey, we now have a computer on every desk that does things that were unimaginable 20 years ago, and they're all connected in this network that gives us instant answers and instant connections. These are miraculous things. And then you find other people who say, you know what? We're still writing code basically by picking out characters one at a time, we still have programs that are laid low when a single bug creeps in, we still have projects that take ten times longer than they should, we need to rethink everything from the ground up.

I don't have an answer between them. My personal temperament is more towards the optimistic. In the end, what you've got is this industry that's been conditioned by Moore's Law, and by its own fantastic financial success, to assume that the curve is always an upward curve, that everything gets better at an exponential pace. That's the experience of the technology industry. You have that smacking up against the reality of human experience, of creativity, of people working in teams. We have these basic human factors, psychology, the limits of the conceptual capacity of the human brain—things that do not move at an exponential pace. They simply don't. They tend to move linearly, if they are improving at all. People in the technology industry are loath to accept that.

CIO Insight: You talk about "the physics of software." What's that all about, and could it help solve some of these seemingly intractable problems?

It's something we don't have now, that some people are yearning for. The idea is that if we're going to turn the creation of software into a true science, we need to first have principles. We need to know the fundamentals and formulas by which software behaves. What are the laws and principles we can count on in creating it? The problem is that software is not a thing, not a preexisting phenomenon of the universe; it's a product of the human imagination. One of the ideas that stayed with me most strongly is that computer science today accepts the file system as if it were some law of nature, and it's not. Every aspect of software is a human construct. The fact that we have created this pile of abstractions, one on top of another, and that it works most of the time—and produces amazing results—doesn't mean the whole enterprise is a fait accompli and must always be that way. So, personally, I don't believe there is such a thing, or could be such a thing, as the physics of software. But accepting that there couldn't be such a thing has implications that are disturbing to people in the software field.

CIO Insight: Many problems with big projects appear to stem from expectations: The bosses aren't realistic, the programmers overpromise and overreach. It seems like a management problem and a culture problem, not just a software problem.

It's all bound up together. Some of it is rooted in the industry's own history and rhetoric, the mythology of forced marches and death marches and Netscape Time—all these things that suggest that if only you push hard enough, you can do it faster. Managers get into saying: "I want it yesterday, the competition had it yesterday, why can't I have it yesterday?"

Another aspect is that, because software is so insubstantial and a lot of managers who are not programmers themselves don't really understand anything at all about it, they don't have the frame of reference that people have when they're building a bridge or a building. When someone is building a skyscraper, there is a gut-level understanding that you have to dig the hole before you can pour the concrete, then you can put up the beams. There's a natural sequence of dependencies in the physical world that you can imagine very easily, and you can see it's going to take time. Software is basically entirely abstract, except for the screens, and the screens are what business people always end up focusing on. The insubstantiality of the product promotes the idea that, hey, why should it take so long? There's nothing there.

Next page: Shifting Ground

1 | 2 | 3