IT Productivity: Measuring the Immeasurable

By Edward H. Baker  |  Posted 01-07-2007

IT Productivity: Measuring the Immeasurable

The productivity numbers released by the U.S. Bureau of Labor Statistics in October 2006 sent a shock wave around the country: After showing robust increases for several years, productivity growth had slowed to an annualized 0.0 percent (though that was later revised up to 0.2 percent). Meanwhile, the trailing 12-month figure sat at a paltry 1.3 percent, the slowest growth since 1997.

What happened? The news of the slowdown, or near halt, to productivity growth brought with it fears that the rapid gains of the past five years were over, and that IT's contribution to productivity growth was perhaps overstated. But wasn't information technology supposed to fuel never-ending productivity increases? Isn't that why we spent all that money on the stuff?

The answer, not surprisingly, is complicated. It's so complicated, in fact, that in the ongoing debate over the impact of technology on productivity, Accenture Ltd., Microsoft Corp. and Hewlett-Packard Co. have created a new trade association dedicated to defending IT's honor. Their Institute for Innovation and Information Productivity, or IIIP, is a nonprofit group endeavoring to redefine how productivity is calculated and measured in an information-based service economy, a critical issue for technology and IT services vendors of all stripes.

Karen Lojeski, the research director for the IIIP, puts the problem in strong terms. "The way productivity is measured today just doesn't make any sense. All these productivity measures are based on manufacturing models, and there's no accurate calculation at the macro level of how to capture productivity in the service sector. And when you have an economy that's 80 percent service, you've got a problem. The idea of trying to understand it as the number of people working times the number of hours worked equals X number of widgets doesn't make any sense in a knowledge-based environment."

The questions that the IIIP is asking—and trying to answer—are ones that have vexed the business community for decades. And they are fueling a revival of the classic debate over IT productivity. What is IT's contribution to productivity? Is productivity even the proper way to measure the potential for economic growth and increases in the nation's standard of living? The answers depend on whom you ask, and on what it is you're really trying to measure. And the answers are critical, as they hold the key to understanding the true value of IT investments.

A Brief History of

Productivity">

A Brief History of Productivity

As it turns out, economists attributed much of the recent slowdown to a general weakening in economic growth: When gross domestic product slows down, as it did in the third quarter of 2006, it's often accompanied by a slowdown in productivity growth, as companies anticipating further gains continue to add labor to the productivity equation without concomitant gains in output. But this time around, the fears of a productivity slowdown seemed to rattle more than a few nerves in the IT community.

The measures ordinarily used to measure productivity, such as labor productivity and even multifactor productivity, involve simply adding up all the known inputs and outputs and then doing the math. In the case of labor productivity, that's just output divided by the number of hours worked. In the case of multifactor (or total factor) productivity, that's all the known outputs divided by all the known inputs. Pretty straightforward, and it generally worked, at least as a rough measure during the Industrial Age, when the inputs and outputs were relatively easily tallied.

But the arrival of the Digital Age has exponentially exacerbated the problem. In the early days of the information revolution, when IT resided beyond the glass wall, IT's contribution to productivity was relatively easy to assess. Take the cutting of payroll checks, for instance: How many laborious hours spent writing individual checks could you eliminate by feeding payroll data into a computer that automatically calculated the amount of the check, and withholding and Social Security taxes, and then printed it? Machines replaced humans, and it was simple to see the return.

By the late 1980s, however, with the advent of personal computers, networks and back-end systems that touched more parts of the business, the math became more difficult. In 1987, Nobel Prize-winning economist Robert Solow wrote, "You can see the computer age everywhere but in the productivity statistics." The problem, analysts decided, was either a lag in the time it took for companies to benefit from their IT investments, or the difficulty of measuring IT's contribution accurately—or both.

Then came the 1990s, which brought about a transformation in IT and how businesses used it. The Internet, the graphical browser, e-commerce, as well as enterprise-level software such as ERP and CRM, brought about massive investments in computers, networking equipment and software. But all that investment only fueled the controversy surrounding just how much IT contributes to productivity gains, and whether it is accurately measured.

In 2001, McKinsey & Co.'s McKinsey Global Institute published a still-controversial report arguing that most of the productivity gains in the 1990s were attributable to just six sectors of the economy: telecom, semiconductors, computer manufacturing, securities, and wholesale and retail distribution. In other sectors productivity was essentially flat or had actually declined.

That led McKinsey to suggest that IT was just one of a number of factors that generated the decade's productivity gains. "The problem," says Diana Farrell, director of the McKinsey Global Institute, "was that all sectors of the economy spent on IT, but because the productivity gains were concentrated in just those six sectors, you clearly can't attribute the growth to IT alone." IT is needed to facilitate productivity gains, she says, but only under the right conditions, which include sufficient competitive intensity and sufficient demand—exactly the conditions faced by those six sectors that contributed so much to the productivity gains of the 1990s.

Erik Brynjolfsson, the George and Sandra Schussel Professor of Management at the MIT Sloan School of Management and director of the MIT Center for Digital Business, offers a different rationale. To explain what happened, Brynjolfsson has developed a concept he calls organizational capital:

"The work I've been doing suggests that most of the benefits from IT come from complementary investments in what we call organizational capital—business processes and other changes in the way companies are organized. For every dollar spent on IT hardware, $10 is spent on this kind of business reorganization.

"What happened in the 1990s was that companies were investing a lot, not only in IT, but also a tremendous amount in organizational capital and new business processes, and those investments tend to take several years to pay off. Go forward a few years to 2001–02, when IT spending dropped, as did investments in organizational capital. The focus was not in adding to organizational capital, but in harvesting what had been done. The way that affects productivity is that the government statistics don't measure organizational capital at all, but they do measure the output that's generated from it. So that is why we had very high productivity, circa 2001–03, as we were reaping the benefits of these investments but not incurring the expense of new investments in organizational capital," he says.

And the recent drop in productivity? Says Brynjolfsson: "If you don't invest in IT or organizational capital, then three, four, five years down the road, you're not going to be in the position to get decent returns. So five years later, 2006, that's more or less exactly what's happened: The investments that weren't made in 2001 are the reason we're not having the comparable level of productivity growth today."

IT and its Discontents

IT and its Discontents

The problem with IT as a source of productivity gains is that those gains, as the McKinsey study showed, are so unevenly distributed. Put bluntly, some companies, and some sectors, are a whole lot better than others at putting IT to work productively. John Parkinson, former chief technologist for the Americas at Capgemini, points out that IT can only boost productivity if it lets you do more with less effort, or you make the time it takes to do the work shorter. But a lot of new technology innovations, he says, "imposed a productivity penalty by forcing people to relearn how they do things, which reduces their productivity for a while. That, I think, is what the McKinsey study showed: They picked a period of time, the 1990s, when lots of relearning was going on, to do the study. Everybody was disrupted by broad adoption of PCs and networks, the Internet, e-commerce. But once we figured it out, the productivity gains kicked in, because you were over the learning hump.

"The second penalty you pay, because technology makes possible things you couldn't do before, is that you now do those things because they're possible," Parkinson adds. "And that adds work to the total amount people have to do." To illustrate the problem, Parkinson points to information-based decision-making. "It used to be that managers didn't worry about gathering mountains of information, populating spreadsheets, building in the models and agonizing over what they told you. You just said, 'yeah, I think we'll go that way.' And you went. But now that process takes a lot more effort. So there's a productivity loss in some parts of business. But that doesn't mean there's an effectiveness loss. It just means that the cost of being right went up."

On a larger scale, much of the investment in IT made by corporations was simply not productive. "There was a lot of "me-too" investment in IT across a lot of sectors since 1995," says McKinsey's Farrell. "Companies saw that their rivals had something new, and so they made these very large investments without recognizing that even within a sector, different competitive strategies mean your productivity could be driven by different things. But making IT investments in areas that aren't driving the bulk of your potential productivity gains is wasted investment."

Measuring the Wrong Things

Measuring the Wrong Things

Perhaps the reason that IT has been much-maligned when it comes to productivity is that its role has been widely misunderstood. Farrell's ultimate conclusion is that considering IT a primary productivity driver isn't really accurate. Yes, it's an enabler, and even, nowadays, a necessity. But as she says, it isn't sufficient. "The real driver of productivity, before, during and after the dot-com boom, is innovation. Only innovation can drive both increases in value-add, and decreases in cost," she says. In and of itself, Farrell maintains, IT can't make a real difference. Instead, it's simply part of the process by which managers innovate—and thus drive productivity gains.

In Farrell's view, innovation begets productivity in three ways: The first involves the development of the innovation itself, whether it's a new product, service or process. The second involves how innovations get disseminated within the innovating company, or among competitors in its sectors, and eventually in other sectors. And the third is how those innovations scale to their optimum use. That's where the growth and profits come from. IT plays an increasingly critical role in this process. "In this context," she says, "one of the reasons IT is such a powerful tool is that it can enable all three of these processes. IT enables many innovations directly—mobile telephony, online securities trading and retail innovation come to mind. And it enables the diffusion of innovation much more quickly because you can replicate IT services much more quickly than you can other innovations. And finally, because IT can scale so well, it can help take innovations to their maximum potential." Still, it's not IT but the innovation that creates the competitive advantage.

Parkinson puts the problem another way. Raw productivity gains, he says, aren't sufficient to compete successfully in the 21st century. "What you want is agile productivity," he maintains. "You want to be able to repurpose the assets of your business as efficiently as possible to stay current with the market. The early 20th-century model of capital efficiency, which is what built corporations and drove the process-focused productivity of the second half of the century, is being rethought around the ability to move all assets, capital, people and information as effectively as possible." Agility, of course, is but another way to describe a kind of perpetual state of innovation, of moving fast enough—through product and process development, and into and out of markets as opportunity dictates—to compete.

That's a consistent theme among the experts and CIOs alike. Atefeh Riazi, the worldwide CIO and a senior partner at Ogilvy & Mather Worldwide, the New York City-based advertising agency, points to growth as the metric that really matters in this environment. "Cutting costs and making your people more productive is critical, of course," she says. "We have a huge mobility program in place which is going to help our people become more productive. But it goes beyond all that. Now it's whether you can get to market faster. Your competitors are coming from places you did not expect. So you have to respond faster, smarter and cheaper. All of these are going to help grow your business. It's no longer just about cutting costs."

Measuring Matters

Measuring Matters

Here's a scenario that demonstrates how the same IT investment can reap wildly different rewards. Two companies in the same industry invest in identical IT systems. The first company has both the innovative culture and the managerial wherewithal to rethink the processes affected by its new system, and revenue and earnings grow by 15 percent. The second company, with a different culture and managerial goals, concentrates on using its new system to cut staff; productivity grows by 3 percent and earnings grow by 5 percent, but revenues don't grow at all. The second company saw productivity gains from its IT investment, but the first company saw the greater benefit.

Given those two scenarios, the goal for CIOs now is to figure out how to measure the real benefit of successful innovation, and of IT's impact on the innovation process, not just the benefit of IT by itself. "In order to advance in a knowledge economy, you need to be innovating on a consistent basis," says IIIP's Lojeski. "Whether you have radical innovations, incremental innovations, process innovations, product innovations, technological innovations, administrative innovations, you need to be able to draw a straight line from the knowledge workers—and the inputs he or she makes use of, including IT—to the effectiveness of the innovation. So what we're trying to measure is the effectiveness of our output, especially when it comes to innovation and value creation. How effective are we as knowledge workers?"

If organizations could get a better handle on the true value of their knowledge-based outputs, Lojeski concludes, they would have a much better basis on which to make decisions about the inputs they use. "The overwhelming majority of executives are very frustrated with their innovation investments globally," she says. "They're reaching for all kinds of new innovation models—R&D, open innovation, outsourced innovation. But in order to face shareholders and say, 'We're providing shareholder value by investing in these kinds of things,' we have to be able to measure that value. If you can't measure what you're doing accurately, it's very difficult to value your investments accurately.

"We don't have the answers yet," Lojeski adds. "But we're thinking that, ultimately, effectiveness will replace productivity as the standard measure of growth for the knowledge economy."