IT Metrics: Feast or Famine

“For me, goals and daily metrics are the key to keeping me focused. If I don’t have access to the right stats, every day, it is so easy for me to move on mentally to the next thing. But if I have quick access to key metrics every day, my creativity stays within certain bounds–my ideas all center on how to achieve our goals.” -Paul Allen

Metrics are a necessity in managing just about anything.  The media is loaded with metrics on GDP growth (or lack thereof), new home sales, unemployment, and presidential approval ratings. Business discussions are laden with metric-oriented acronyms: ROI, ROE, EBITDA, NAV.

Why, then, does the IT profession struggle so badly with metrics? The IT organizations at a majority of companies we’ve come across typically fall into one of two camps:  Either they are managing against too few metrics (or none at all) or they are overwhelmed and trying to manage to too many metrics. 

Too Few – “Famine”

Managing by too few metrics usually is indicative of one or more situations:

•    There is the pendulum effect—over-simplifying in response to an earlier feast of overwhelming data that soured management on metrics.
•    Sometimes leadership doesn’t know where to start.
•    There is uncertainty about how to collect the necessary data.
•    Particularly in this economic environment, reporting is sometimes considered too expensive, particularly if its value isn’t blindingly obvious.

For example, one Fortune 500 financial services company I’ve seen reports solely on project financials across the enterprise.  They tracked details such as: “How much has the project spent? What was the budget? How much does each person cost?”  However, nowhere in that spectrum are metrics that address how much value the project has achieved for the money spent, how it relates to company strategies, or anything about benefits.  The set of metrics this company settled for does not allow for any true prioritization or decision-making that could be useful to the overall business direction.

Too Many – “Feast”

What’s going on when the IT organization is overwhelmed by metrics?  In a decentralized organization, often no one bothers to establish agreement about measurement. Instead, a culture of satisfying “customer” requests breeds a multitude of ad-hoc metrics and reports.

Such was the case at a multinational Fortune 500 firm. The IT department had trouble agreeing on a single set of operational metrics because they feared the wrath of the next executive whose request couldn’t be perfectly filled.  As a result, IT was on the hook more than 100 different reports, most run just once or twice a year.  

In other cases, getting people to agree on the “right” metrics is virtually impossible. At a Fortune 100 insurance company the leaders of two huge technology units were never able to agree to a common set of metrics out of different systems.  They both ended up at the mercy of their business counterparts who set a messy metrics agenda.