Opinion: The Perils of Project Planning

In my last column, I wrote about the challenges we face when trying to create a useful estimate of the effort required to complete a software development project and the methods available to do so. As challenging as this is, it's just the start of our journey to a predictable project outcome. Now we have to take that estimate and turn it into a plan that will allow us to "manage" (from a root word that meant "keep a hand on") the work.

Let me start with a couple of important observations:

First, for small projects, and experienced, self-organized teams, a formal plan may not be necessary. Such teams have a good sense of how long it will take them to finish and are good at assessing what they can commit to. In these circumstances, formal plans may serve other roles (perhaps as a framework for collecting information about what actually happens or as a template for reporting progress). But they won't enhance the project team's work. That doesn't mean that smaller projects don't need to be planned for.

Second, the variables and constraints we work with when we plan are not independent. As I add resources to a team, I add coordination overhead (and maybe other kinds of overhead as well), so the proportion of non-contributing resources goes up, and so do my costs.

There are also intrinsic rate variables to consider. I can't get 100 days of work done in one day using 100 people if some of the work takes a minimum of 5 days. (This is a problem of "irreducibility," and it often gets overlooked. Nor is it always obvious to the estimators and planners, or even to the team members. (If you're interested in the theory of irreducibility, look up the work by Greg Chaitin at IBM Research.)

Absolute duration also matters: The longer the project lasts, the higher the probability that I will have to adjust its scope and the more review and re-planning effort I will need.

We also want, wherever possible, to achieve "resource leveling": keeping the project team the same size throughout the project, which helps achieve the best utilization without making some people split their time across more than one team, which generally reduces their productivity within any one team.

And finally, we have to factor in differences in ability amongst the team members. Ideally we would like to assign the work in amounts that matched exactly an individual's ability to deliver. Unfortunately, we seldom have enough knowledge to do this perfectly, and people change enough over time to make past performance an uncertain predictor of the future.

So planning is really an exercise in tradeoffs and compromises. Planners (and their tools) try to get the "best" answer from each situation by "fixing" one or more of the variables and optimizing against the rest. So we fix duration and vary effort and scope. Or fix scope and vary effort and duration. The most sophisticated tools run through millions of combinations of possible tradeoffs and propose one or more "optimum selections" according to defined characteristics like cost, delivery time and resource utilization.

Then someone makes a selection and you're off to the races.

Next page: What Can Go Wrong