Business Intelligence: Meet the Data WranglersBy Tony Kontzer | Posted 05-06-2011
Business Intelligence: Meet the Data Wranglers
Regardless of the business sector in which your company operates, chances are you're struggling with data overload. In fact, the wrangling of data has become a huge area of focus for most IT organizations.
"Everybody is drowning in data, and we haven't even seen a massive amount of data yet," says Hossein Eslambolchi, CEO of 2020 Venture Partners, which provides IT consulting service to venture capital firms. Eslambolchi is author of 2020 Vision (Silicon Press, 2006), an exploration of how IT will transform business over the next decade. He expects data to become a growing challenge over the next 10 years, as wireless carriers beef up networks to support data consumption via super-fast broadband connections.
Some companies are doing what they can today to squeeze the most value out of the massive amounts of data they're shepherding. What follows are tales of how four very different enterprises have made headway in wrangling data. For more on this topic, read the article "Business Intelligence: Five Tips for Managing Data."
ComScore: Analyzing Data Where It Sits
Few companies can claim that data are more important to their business than ComScore, which monitors the online behavior of some 2 million consumers around the world. The company collects information ranging from general online activity--such as sites visited--to granular details, such as mouse movements and clicks. It then uses that information to provide projections that companies use to plan every part of their digital strategies, including marketing, sales and product development.
The amount of data ComScore accumulates has been growing rapidly, forcing the company to constantly revamp how it crunches that data to assemble projections. ComScore captures 120 million pages a day in analyzing online transactions, and its database stores 2 trillion records totaling 147 terabytes. The company had been pulling data into individual applications for analysis, an approach that had become increasingly inefficient and time-consuming, says Scott Smith, vice president of data warehousing.
"At the end of the day, our whole business model is predictive analytics," says Smith. "We needed to get faster in coming up with projections."
To speed its data analysis, ComScore recently went live with a platform that combines Sybase's IQ analytics server with software from Fuzzy Logix, enabling the company to analyze information in the database. Doing so, Smith says, is paying off for ComScore on a number of fronts:
- It no longer has to move massive data files--sometimes as large as 500 GB--from the database to any number of applications.
- Programmers no longer have to write complex analyses in SQL, saving countless hours.
- Vast amounts of bandwidth that had been devoted to the movement of data between the database and the apps have been freed up for other uses.
- Because the new setup prevents data from being used repeatedly after it's been removed from the database, ComScore's projections are based on fresh, accurate and accessible data.
"It gives me a lot more confidence in terms of how the analysis is done because we know what the state of the data is," Smith says. "Previously, we would have had to extract the data and prep it to do all that analysis. Now, we're doing it all directly in the data warehouse."
SecureAlert: Tapping Data to Stay Ahead of Crime
SecureAlert can't afford to let information go unanalyzed. Public safety is at stake. The six-year-old firm uses ankle bracelets transmitting GPS coordinates to track probationers and parolees for law enforcement agencies and bail bondsmen. Its business relies on its ability to react quickly and decisively to real-time data. And it has a lot of data to react to. Each of the ankle bracelets worn by the 10,000-plus offenders tracked via SecureAlert transmits data every 10 seconds, 24/7, to the tune of more than 2 terabytes every year, says Steve Florek, vice president and managing director of offender insights and knowledge management.
It's not the kind of data that can be dropped into a data warehouse and analyzed later. SecureAlert has to extract everything it can in real time so it can react and dispatch law enforcement if necessary.
That capability is provided by ParAccel's analytics-intensive database engine, which provides the company with insight into exactly what each offender is doing at a given moment. For example, a parolee who wanders out after a court-imposed curfew can be told via the ankle bracelet's built-in speaker that police will be dispatched if he or she doesn't return home.
Likewise, SecureAlert could trigger the alarm on an ankle bracelet worn by a person on probation for domestic violence if that person gets too close to the victim's home. ParAccel also enables SecureAlert to identify behavior trends that might otherwise go unnoticed to help predict potential crimes. Alluding to the film Minority Report, Florek says, "We want to be the department of pre-crime."
For example, the analytics solution recently was able to deduce that a sex offender in Oakland, Calif., had been milling about a certain intersection at 2 p.m. every day. The software brought this to the attention of the SecureAlert staff, which discovered that the intersection was the site of a new school bus stop. The company dispatched law enforcement to intervene before any offense could be committed.
Florek says the ability to react immediately to GPS data, spot trends in offenders' movements, and provide real-time responses to emergency situations has been a "sales differentiator." He adds: "Instead of telling you yesterday's news, we're actually able, in real time, to detect criminal activity and get ahead of it."
Distribution Market Advantage: Data on Demand
Like many companies dealing with fast-growing data volumes, Distribution Market Advantage was overwhelmed by the task of extracting value from its information. A cooperative of 14 regional food distributors intended to help those distributors compete with larger national rivals, DMA was unprepared for the data-analysis business. The solution the firm chose for its data needs simply wasn't up to the task, according to Jim Szatkowski, vice president of data services.
To stay focused on its core business, the company several years ago had outsourced its IT environment to iTradenetwork, a cloud-based supply chain management solution for the food service industry. With some 150 supply-chain managers feeding a data mart that had grown to an unmanageable 250 million rows of records, DMA decided to jettison the cumbersome Access approach and turn to the cloud once again, this time subscribing to PivotLink and its software-as-a-service business intelligence platform.
The move turned out to be transformative. Suddenly, food distributors such as Ben E. Keith Foods, Gordon Food Service and Reinhart FoodService could rest easy, knowing that restaurant chain customers such as Chili's, P.F. Chang's and McCormick & Schmick's could easily get to the data they needed, including invoicing records, delivery schedules and cost analyses.
"Customers want to be able to go through invoice details to make decisions about supply purchases," says Szatkowski. "PivotLink gave us a quantum leap forward in terms of the ability to harness this data."
The impact of DMA's improved data analysis is visible throughout its supply chain environment. For instance, in the past, customers had to wait for DMA to assemble requested reports. Today, customers can generate 80 percent of those reports themselves by accessing the data and report-building tools directly from PivotLink. Szatkowski notes that one customer, nursing facility operator Daybreak Venture, has used PivotLink to pinpoint how much it is spending per patient on meals, enabling it to tweak its purchasing habits and reduce costs.
By tapping DMA's data-analysis tools, distributors have improved the accuracy and coordination of orders, thereby improving deliveries to customers. One distributor was able to reduce its number of deliveries to a group of eight stores from 35 per week to 16 per week, Szatkowski says.
Overall, DMA's distributors and customers are saving an average of $50 on every invoice because of the cooperative's improved data-analysis capabilities, according to Szatkowski. And DMA isn't stopping there. It continues to add integrations from PivotLink into other areas of its business, leading to additional capabilities, such as a price-verification component. This guarantees that customers that negotiate volume-pricing deals with distributors receive invoices reflecting those prices. "We're much more in tune with how our business is performing."
Southeast Texas Medical Associates: Patient Data
Health care is "way behind the curve" when it comes to data management, says Dr. James Holly, CEO of Southeast Texas Medical Associates (SETMA), a family medicine and gerontology practice.
Holly, who also acts as de facto CIO, has refined the use of data at his 24-physician practice over the past two years. Having started using electronic health records in 1998, Holly oversaw SETMA's steady development of a tightly integrated 60-application IT environment. Despite all the cutting-edge capabilities those apps gave the practice, the value of all the resulting data--"millions upon millions of pieces," according to Holly--was undermined by the lack of a powerful data mining tool.
It could take as long as 36 hours to assemble and analyze the data needed by doctors from the maze of applications. "We knew what we wanted to do, but we were incapable of doing it," says Holly. "We had to settle for much less."
In summer 2009, the organization deployed IBM Business Analytics software and created a data warehouse. Holly says the software's ability to pull information from SETMA's application portfolio into the data warehouse and perform real-time analysis reduced that 36-hour analysis window to a mere 15 seconds.
The resulting insights have enabled SETMA to raise the bar on patient care. For example, doctors were able to recognize an important and disturbing trend: Many diabetes patients were losing control during the eating-intensive holiday season. These patients also were less likely to visit their doctor and have important tests performed during that time period. Because this can lead to unnecessary complications, SETMA started asking diabetes patients to sign contracts promising to be seen over the holidays. As a result, "We've seen a remarkable improvement in our ability to intervene in that population's care," says Holly.
SETMA's improved use of data also enhanced the quality of post-hospitalization care by ensuring that when a patient is discharged, everyone--the patient, hospital staff and SETMA physicians--is armed with the right information. This can include details such as the patient's medications and dosages, or specifics about the home-care environment to which the patient is returning.
The result has been a 22 percent decline in hospital re-admissions, translating to significantly reduced costs, enhanced care and an improved patient experience, according to Holly. "We wouldn't know whether what we were doing was working if we didn't have the business intelligence software," he says
Delay at Your Own Risk: The Scale Will Kill You
Make no mistake--for every company that's extracting value from its data, there's another with its head in the sand. "A lot of companies aren't paying attention to this," says 2020 Venture Partners' Eslambolchi. "They're just paying attention to day-to-day operations." The way he sees it, companies have an opportunity to reduce costs and increase revenue with added services by getting on top of their data. It's also a matter of survival. Says Eslambolchi: "You're going to reach a point where the scale is going to kill you."