Business Intelligence: The Real Challenge

Businesses have been accumulating data in digital form from business operations for more than 40 years now. In parallel with this accumulation, tools for organizing, categorizing and analyzing this data–turning it into the kind of context-rich information that can drive decision making–have been growing in capability and sophistication.

If you believe the vendor stories today, we can “empower” everyone in the business with the information tools they need to make better, faster operational and strategic decisions and thus please both customers–because we know who they are as individuals and can anticipate what they want and need–and stakeholders–because happy customers make for a profitable business. There are even credible “proof points” to back up the claims, such as case studies of businesses that do this really well.

Information-driven businesses do exist. They do work. And some of them even do better than the rest of the market they compete in. Shouldn’t we all make use of the mountains of data in our warehouses and match the performance of these pioneers? Isn’t it time for “Enterprise BI”?

If only it were that easy.

In the more than half a century since Peter Drucker and others introduced the idea of an information-driven business, a lot of practical issues have surfaced alongside the piles of data we have accumulated.

Three in particular stand out as major impediments to pervasive and effective BI:

• Few organizations have consistent definitions for all of the data they have accumulated, making reconciliation a necessity before analysis can be performed reliably. This “master data management” process is a late entrant to the BI party, and the tools for it aren’t all that great yet. That’s because this is a genuinely hard problem and cleaning up four decades of metadata indifference is expensive. Lacking the appetite for the required discipline, most of us (or our business sponsors) concluded that the cleanup wasn’t worth it in the general case, compute cycles are pretty cheap, and our analysts could always figure things out if we ever needed to get a “single version of the truth” together.

Which brings us to the second issue: audience.

• It can be argued that you get the most value from enterprise BI when it’s accessible by and integrated with the work of your frontline staff and managers. The people who touch your customers and critical business partners make myriad decisions every day that potentially could be influenced by better information. Instead, BI has generally focused on a much smaller group of specially trained “data analysts” who use sophisticated (and often pretty “user-hostile”) tools to analyze business performance and make “reports” available to managers to take (after the event) corrective action or to inform the strategy-creation process.

As valuable as this might be, we’re not going to get to pervasive BI along this route–which arguably misses the whole point of individually better-informed, real-time decision support–and following it leads us to the third issue: comprehensibility.

• Ideally, information would be presented in a context and form that is instantly recognizable as relevant and immediately absorbed into the decision-making process. That way, “intelligence” influences the decision being made if it should do so (and not if it shouldn’t). We seldom get either capability in today’s BI platforms. The first requires our BI platform to have “situational awareness” so that information can be selectively filtered for relevance in a dynamic, complex, hard-to-predict pattern. The second requires the platform to present information in a form that is in part tailored to the cognitive preferences of the user–who could be anyone, including people providing managed services to your business and thus who don’t work for you and are outside of your control. These are hard problems for the BI platform vendors, but harder still for a business’s organizational design, role design and human capital management process to address.