Why Dark Data Shouldn’t Be Taken Lightly

More than 80% of enterprise data is considered to be dark, or data that is captured and stored but never used. Dark data contributes to an analytics deficit — meaning enterprises have a surplus of data, but a shortage of insights from it.

This represents a latent opportunity for businesses to use more existing data to serve customers, compete, and operate better.

Read more: Are You a Data Hoarder? The Dangers of Data Hoarders in Business

What Is Dark Data?

The term “dark data” refers to data that enterprises capture and store, often as part of their regular business processes, that they then fail to use. Hidden and usually unstructured, dark data is expensive to store and secure, but most companies do so for compliance reasons.

Hidden and usually unstructured, dark data is expensive to store and secure.

There are many reasons why this data goes unused. Some of it holds little value or is redundant. A lot of dark data does have value, but enterprises haven’t developed the strategy, business processes, or IT processes to extract and analyze it.

What Is the Cost of Dark Data?

The costs of dark data include loading, updating, storing, and managing unused data. This consumes IT personnel time, storage space, and CPU cycles. These time and infrastructure resources could be better spent on higher-value work.

What’s more, unused, old, and redundant data can create vulnerabilities within your infrastructure.

Read more: What Is a 3-2-1 Backup Strategy?

What Should Enterprises Do?

The most successful enterprises identify and extract value from more of their data assets. They also identify and reduce the cost of managing data with little or limited value.

With unified analysis, dark data can offer a deeper customer view and new selling opportunities.

For example, different parts of a large financial services organization might engage the same customer with multiple services — including mortgages, life insurance, and commercial banking.

Those divisions might store customer information that has no direct value in isolation, so it goes untouched. But with unified analysis, this information could offer a deeper customer view that might identify new selling opportunities.

How Can IT Help?

By identifying and moving historical records to the cloud or software utilities like Hadoop, you can reduce your storage cost. You also could create new analytics insights by correlating that data with other data in Hadoop.

The first step is to understand whether and how different data sets — databases, tables, and columns in a data warehouse — are used, by whom and at what cost. These metrics help IT make better decisions about whether to keep data and how to use it.

At a strategic level, the key is for enterprises to start thinking about how to make better decisions with what they already have.

Read next: Data Collection Ethics: Bridging the Trust Gap