ROI: Why Don't More CIOs Measure ROI After a Project Is Up and Running?By Allan Alter | Posted 04-01-2004
It seems like a no-brainer: After putting in all the time and effort required to see if an IT investment will produce a worthwhile ROI, surely it's logical to find out whether you've actually received it after the project is over. "If you don't learn, you can't improve. Seems obvious to me," says Tom Pisello, CEO of Alinean Inc., an Orlando, Fla., ROI consulting firm. Yet only 46% of companies calculate the ROI of an IT investment after projects are completed. And judging by our interviews with seven CIOs, even these companies don't do so in all cases. The reality is that ROI remains primarily a tool for justification, not for measuring actual value.
Why? Surely executives should want to learn whether a system costs more than expected, or isn't delivering the anticipated benefits, as did Denise K. Fletcher, a veteran CFO who is now managing director at the FA Group. Surely they'd like to use this information to make decisions about future investments, or to see if they need to adjust a business process or the system itself so the latter can deliver as promised, as does Robert Reinckens, CIO for the Americas at Coty Inc., the New York City-based fragrance and cosmetics manufacturer. And what CIO wouldn't like to boost their credibility by hitting their ROI target?
One reason: It's hard to act on logic when fear gets in the way. If managers are afraid they'll be embarrassed, or put at risk if they miss their ROI numbers, they won't be eager to follow up. Roger Santone, the CIO of USF Processors Inc., a Dallas company that helps retailers and manufacturers manage product recalls, returns and salvage, says IT and business executives at his company do not calculate ROI on projects after they've been implemented. "Deep down, the reason people don't do it is they don't want to be the bearer of bad news. You are laying it on the line and saying, 'The decisions I made weren't correct.' That's a tough call, but it needs to be done or you never learn from your mistakes."
It's also hard to do the logical thing when the desire to calculate the actual ROI isn't strong enough to overcome its difficulties. CIOs told us, and ROI consultants have heard them say, that they simply don't have the time to do a post-facto analysis, or that they only do it for the most important, expensive or visible systems. They say their companies lack discipline, their IT people lack know-how, or the information needed to calculate ROI is too difficult or costly to obtain. Whether these obstacles are genuine or just a cover for avoiding professional embarrassment, managers need to be strongly motivated to overcome them. That motivation isn't always there, at least not for every IT investment, and for all the people who need to be involved. "The main hurdle is putting real purpose into doing the post-implementation audit. Unless information is expected to be used to make decisions, little incentive exists," says Bob Benson, a principal with The Beta Group, an IT management consultancy in St. Louis, Mo.
At Coty, for example, the IT organization doesn't perform an after-implementation ROI on critical base systems like ERP because these are core applications required to run the business. "It's a given: If you want to get into the dance, you have to have it," says CIO Reinckens. He will, however, analyze ROI on applications built upon those systems, such as an advance planning and scheduling system, to verify the expected business benefits and help decide whether to spend more on such system enhancements or additional IT investments. And in many organizations, avoiding cost overruns and making sure the system functions is often, if not always, sufficient. At the City of Las Vegas, an expensive ERP system would receive an after-implementation ROI analysis performance review, but less costly back-office systems and narrowly focused applications that are meant to support ERP processes, and that wouldn't catch the public's attention, don't receive the full treatment. "I just want to make sure it works," says CIO Joseph Marcella.
Meanwhile, we found little evidence that ROI accountability was a consistently high priority among CEOs. It doesn't usually factor into how CIOs are rewarded, Pisello says. "CIOs are typically measured on cost, not performance, which is one of the biggest factors in why CIOs don't measure ROI." And while all the CIOs we spoke with said they needed to project ROI to receive funding, only Chris Loehr, VP and IT manager at Oklahoma City-based Local Oklahoma Bank, stressed that his CEO, Edward Townsend, consistently held him accountable for achieving ROI. "Did the investment make the profit or recover the costs you said it would? If not, you have a lot of explaining to do," Loehr says.
Perhaps the reason more CEOs aren't like Townsend is that they don't really believe in the original ROI figures anyway. Indeed, almost a third of this month's survey respondents say the business side at their companies doesn't believe their ROI projections. As Beta Group's Benson put it: "Not every senior manager really believes the details of the pre-implementation business case ROI anyway, especially for projects with 'strategic' or 'innovative' content. They certainly will not believe the post-implementation ROI any more than they did the pre-implementation ROI."