Shedding Light on the Problem of Dark Data

 
 
By Karen A. Frenkel  |  Posted 05-13-2016 Email
 
 
 
 
 
 
 
 
 
  • Previous
    Shedding Light on the Problem of Dark Data
    Next

    Shedding Light on the Problem of Dark Data

    More than 80 percent of enterprise data is considered "dark,” defined as data that is captured and stored—yet never used.
  • Previous
    What Is Dark Data?
    Next

    What Is Dark Data?

    Dark data refers to data that enterprises capture and store, often as part of their regular business processes, that they simply fail to use. Hidden and usually unstructured, dark data is expensive to store and secure, but most companies do so for compliance reasons.
  • Previous
    Why Does It Go Unused?
    Next

    Why Does It Go Unused?

    There are many reasons why dark data is unused. Some holds little value. A lot does have value but enterprises haven't developed the strategy, business processes and/or IT processes to extract or analyze it.
  • Previous
    Why Is Dark Data Important?
    Next

    Why Is Dark Data Important?

    Dark data contributes to an "analytics deficit,” meaning enterprises have a surplus of data, but a shortage of insights from it. There is a latent opportunity to use more of their existing data to better serve their customers, compete and operate.
  • Previous
    What Is the Cost of Dark Data?
    Next

    What Is the Cost of Dark Data?

    The costs of dark data include loading, updating, storing and managing unused data—which consumes IT personnel time, storage space and CPU cycles. This time and infrastructure could be better spent on higher-value work.
  • Previous
    What Should Enterprises Do About Dark Data?
    Next

    What Should Enterprises Do About Dark Data?

    The most successful enterprises identify and extract value from more of their data assets. They also identify and reduce the cost of managing data with little or limited value.
  • Previous
    Identifying and Extracting Value From Dark Data
    Next

    Identifying and Extracting Value From Dark Data

    Different parts of a large financial services organization might engage the same customer with multiple services, including mortgages, life insurance and commercial banking. Those divisions might store customer information that, in isolation, has no direct value so it goes untouched. But with unified analysis, it could offer a deeper customer view that might identify new cross-selling or upselling opportunities.
  • Previous
    How IT Can Reduce the Cost of Dark Data
    Next

    How IT Can Reduce the Cost of Dark Data

    By identifying and moving historical transaction records to the cloud and/or Hadoop, you can reduce your storage cost. You also could create new analytics insights by correlating that data with other data in Hadoop.
  • Previous
    Problem Solving
    Next

    Problem Solving

    The first step is to understand whether and how different data sets—databases, tables and columns in a data warehouse—are used, by whom and at what cost. These metrics help IT make better decisions about where to place their data and how to use it.
  • Previous
    What Else Can Help?
    Next

    What Else Can Help?

    At a strategic level, the key is for enterprises to start thinking about how to make better decisions once they make new combinations of data with what they already have.
 

For enterprises to keep up with the impending surge of data—most of which will be unstructured—they must first understand what dark data is and how it will affect their operations. Gaining that knowledge is the first step toward developing a set of data management processes that will "light up" and extract value from more data. There is a pressing need to do so. Today, 80% of all data is dark and unstructured, John Kelly, senior vice president and director of IBM, recently told attendees of his company's third Cognitive Colloquium. "We can't read it or use in our computing systems. By 2020, that number will be 93%." The annual IBM conference explores the changeover from linear, von Neumann computing to a compute architecture that better mimics the working of the human brain, like IBM's Watson. In this slideshow, Kevin Petrie, senior director of Product Marketing at data management firm Attunity, defines dark data and offers insight into why dark data goes unused, how to identify it and—most importantly—how to begin to put it to good use.

 
 
 
 
 
Karen A. Frenkel writes about technology and innovation and lives in New York City.

 
 
 
 
 
 

Submit a Comment

Loading Comments...