Getting a Handle on Data

Today, all systems lead to big data. But here’s a thought: “Just because data is available and can be fairly easily acquired and disseminated by the business and IT, does not mean that it should be,” said Scott H. Schlesinger, principal in the IT Advisory Service at consulting firm E&Y.

In fact, hoarding the entire data stream may create new problems and it doesn’t necessarily improve performance. “The just-in-time insights that today’s data-driven executives demand are impeded by the sheer volume of data being stored, yet is truly not needed to drive analytics,” he added. In addition, massive amounts of data, particularly personal data tied to customers, can put a business at risk.

The answer (or at least a significant piece of it)? Data lifecycle management (DLM). Although the concept is nothing new, it’s increasingly important to do it right. What’s more, the concept is continuing to evolve and CIOs must tune in, especially as data streams multiply as a result of legacy data, point of sale data, clickstream data, social media data, M2M data and much more.

“The tried-and-true enterprise data warehouse (EDW) has been the cornerstone storage repository for most organizations for more than two decades. However, as time has passed and data levels have reached epic proportions, EDW capacities are being exhausted and load processing has slowed to a near halt,” Schlesinger pointed out.

As the cost for traditional data repositories spikes, organizations must find ways to offload data, through emerging data solutions such as Hadoop and comprehensive approaches such as SAP HANA. It’s also critical to examine data retirement strategies.

Unfortunately, for many organizations, all of this represents a very bumpy road. Too many CIOs and other technology leaders lack a fundamental understanding of how, when and why to migrate a legacy data environment to Hadoop or other low cost data repository, Schlesinger said. There is also a lack of knowledge about how to best integrate and optimize the overall data and analytics environment for enterprisewide analytics.

The bottom line on the front lines of data? In an era where everything is driven by data, CIOs need to move beyond traditional business intelligence and embrace end-to-end platforms that can handle structured, semi-structured and unstructured data in a cost effective way and ensure that it provides, as Schlesinger described it, “speed-of-thought analytics.”

Samuel Greengard, a contributor to CIO Insight, writes about business, technology and other topics. His latest book, The Internet of Things (MIT Press), was released in the spring of 2015.