The Big Data Disconnect Between IT and Business
According to Wikibon’s 2013 Big Data Analytics Survey, respondents, on average, realized only $.55 in return for every dollar invested. They ultimately expect to realize a return of $3.50 per dollar invested over the next 3 to 5 years.
On the IT side, 54% reported complete success. On the business side, only 18% feel the same way.
What accounts for the gap in perceived success between IT and business? They have different criteria for success.
For IT departments, the goal is to get the technology installed, deployed, fault tolerant, and up and running.
Whether the software is Hadoop, a relational database, or something else doesn’t really matter to the business side. It wants answers to questions and insights from all that data to identify new markets, improve customer service, solve business problems and so on.
Asked whether they have shifted a workload from a mainframe or traditional data warehouse to Hadoop, 61% of respondents said they had. That figure is expected to increase dramatically over the next 3 to 5 years.
The balance of power between Hadoop, traditional data warehouses and relational databases is starting to shift, according to the Wikibon study. “EDW [enterprise data warehouse] is no longer the center of the universe,” says Jeffrey Kelly.
The main kinds of workloads being shifted are data transformations. Companies must normalize data from transaction and financial systems, for example, so that they are in the same format and then analytics can be performed, possibly at one-tenth the cost.
All industries will be in the data business one way or another, whether they know it or not. John Deere, for example, is equipping tractors with sensors and analytics capabilities to help farmers understand what crops to plant and when and where.
Hadoop, a 10-year-old open source framework originally developed to support distribution for a search engine project, is relatively complex. “There are not a lot of IT personnel who understand it and how to use it,” Jeffrey Kelly says. “The skill gap is a big one.”