Data Architecture 2002
Can your employees get the data they need to do their jobs well? Is that data presented consistently and clearly? Time and time again, the 665 respondents to our survey link good data architecture practices not only to employees' ability to access important data, but to whether the company is achieving its strategic goals. And both easy access to data and strategic success are strongly correlated to the involvement of top executives from both IT and business in making decisions about data architecture.
If you don't have your data architecture house in order, our survey suggests, you're probably hurting. Executives at successful companiesthose that were successful or extremely successful at reaching strategic organizational goals last yearare more likely than those at less successful companies to say their current data architecture helps them respond quickly to changing business conditions and customer demands.
A number of good practices are being used by companies whose executives say their employees are extremely satisfied or satisfied they can access the data they need to support the strategic goals of the organization. These companies are much more likely to have a formal data architecture plan and data architects on board. And their business units are far more involved in helping to develop requirements for the organization's data architecture.
Oakwood Worldwide Inc., a Los Angeles firm providing temporary housing for relocated employees, once had a system that fit the company's needs like a glove. Two thousand employees in 200 offices around the country relied on a custom HP 3000-based system to match customers with vacancies. Have a client who needs to find a three-bedroom, non-smoking apartment in Sacramento that's near a park? Oakwood's system could help its staff find several to choose from. And it could do it reliably in just a few secondsan important consideration, since it only took eight days of vacancy to wipe out the profits from a two-month rental.
Then those needs changed. Corporate clients wanted to tap into the database so they could stay on top of their relocation spending. Oakwood wanted to analyze all this sales and real estate data so it could uncover opportunitiescould they, for example, boost profits if they offered units with Asian-style kitchens in Washington D.C.? If Oakwood's staff could analyze that data, they could figure out which units and features appealed to different demographic groups. Then salespeople could target Japanese or Korean companies, for example, and the property managers could make modifications. New data analysis tools could do all thatif those tools could utilize the data in the company's computer. But there was the rub, says CIO Ric Villarreal. "I cannot apply any new analytical tools to the data because I don't have access to [the database in] real time," he says. And that would require modifying five million lines of COBOL.
Building a flexible architecture that can integrate existing corporate data and applications while accommodating new business needs and systems is an enormous challenge. Yet companies are finding it is a business necessity and that they can overcome the obstacles incrementally by completely changing applications and databases when they must, and building bridges between legacy architectures and new needs and software when they can.
Companies seldom have complete data architectureswhich comprise the structure of all corporate data, the relationships among the data, and the implementation of data structures in applications and databases, says Paul Harmon, senior consultant for the distributed computing architecture advisory service of the Cutter Consortium, an IT advisory firm in Arlington, Mass. Mergers and acquisitions force the assimilation of dissimilar data architectures, and short deadlines to show financial results often push managers to cobble together the previously separate systems. "About the time they figure out a nice, clean, logical way to merge the databases is about the time management buys another company," Harmon says.
Some organizations have enough operational and organizational stability to sustain a single architecture. The University of Miami has used the same integrated database management system, Computer Associates' IDMS, for 20 years, almost as long as the tenure of Dean, Vice President and CIO M. Lewis Temares. IDMS centrally controls and enforces a data model used by the university's applications, which track students, manage admissions, determine financial aid, create class schedules and support fundraising.
Still, the data architecture has shortcomings. "You have to practically be a programmer to get [information] out," Temares says. Two decades ago, there were no graphical interfaces or high-level tools for university administrators to extract data and create reports. Now, the IT department is providing Web portal front-ends to give administrators easy access to all the information they need to sort through student demographics for marketing purposes, or look at course preferences to plan next year's teaching assignments. Temares is already planning a transition to another database, probably IBM's DB2, at an expected price of $2 million to $3 million.
Creating an organizationwide data architecture is an expensive proposition. A few years ago, the Jersey City, N.J.-based Pershing division of Donaldson, Lufkin & Jenrette Securities Corporation, a Credit Suisse First Boston company, decided it needed a more rational data architecture. Services demanded by consumers, such as consolidated statements, as well as new SEC requirements led Pershing to consider redesigning the structure of its data on customers, accounts and securities trades. But there was a problem: The new architecture would cost tens of millions of dollars and be risky to tamper with.
"We would have to modify every legacy system to use the new data [architecture]," explains director of database technology Albert DiGiovanni. "And the legacy stuff is importantit continually runs the firm." Many of the original business systems run on a mainframe and were built with COBOL, CICS and VSAM. So the company is first mapping out the legacy data, defining additional data required by business needs and storing the extra information separately in DB2 databases. The approach allows Pershing to evolve existing systems while making them compatible with new applications and tools.
"You won't be able to build new all the time," notes DiGiovanni, "so you have to find out how to make the connection [between old and new systems] and then leverage the new data structure."
Managing data architecture changes is challenging enough when planned, but can be harder if end-users unexpectedly force modifications. Another food chain, Red Robin Gourmet Burgers of Greenwood Village, Colo., tries to keep a stable data architecture so data is entered only once and then propagated throughout the system. The group within Red Robin that's responsible for building new restaurants wrote its own Microsoft Access management tool. Now it wants the software integrated with the company's financial application, and Red Robin CIO Howard Jenkins wants to proceed carefully to ensure that the data models are compatible. "Clearly we have to integrate [the tool] or replace it with something easier to integrate. We don't know which yet."
No, it isn't easy to create and stick to a good data architecture. Still, CIOs keep chipping away at the problem, doing what they feel they can.
And effective efforts can pay off in the future. The inflexible architecture at Oakwood is on the way out. A new architecture, supported by an Oracle database and running new third-party applications, will let employees, for the first time, use new tools that will enable them to better analyze data from their operations, and find new ways of increasing profits.
"If I do it right, the architecture makes the processes easy to program or develop, gives the data integrity, and lets me go after the data in lots of different ways," Villarreal says. At least, it will for now.Erik Sherman is a freelance writer in Marshfield, Mass.
The results are available in Adobe Acrobat PDF format. To download the free Adobe Acrobat Reader plug-in, click here.
Data Architecture (PDF Format)
- Data Architecture (MS Powerpoint)
Conclusion 01: Satisfaction
Are employees satisfied they can get the data they need, and are they satisfied with its quality? The answers to these fundamental questions not only seem correlated to one another, but to company success as well. But most IT executives aren't satisfied with how well their organizations are implementing a data architecture and ensuring compliancewhich may be why companies aren't seeing all the benefits of a data architecture.
Little more than half of respondents (54%) say their employees are extremely satisfied or satisfied with their ability to access the data they need. More successful companies are more likely to be satisfied with data access, 60% versus 40%.
Those more satisfied with getting data are also happy with data quality (77%), while those dissatisfied with access are rarely pleased with data quality (24%).
Satisfaction with issues related to the data architecture plan is clearly correlated to company success. Implementation of, adherence to and satisfaction with the plan itselfissues separate from merely having a planare all greater in more successful companies. For example, the more successful companies are extremely satisfied or satisfied with adherence to the company's data architecture plan, 47% versus 30% for those less satisfied. However, such low satisfaction overall probably means IT needs better strategies for creating, implementing and ensuring compliance with its plans.
Conclusion 02: Pros and Cons
Data architecture may seem esoteric, but the benefits of doing it well are real. Both easy data access and organizational success are related to employee satisfaction with data, although only one benefit is enjoyed by more than about half of our respondents. Meanwhile, financial losses due to architecture problems can be substantial, though successful firms report fewer negative effects.
Our respondents point to a variety of business benefits they derive from their data architectures, though none stand out. Fifty-three percent of respondents say their current architecture is helping their organization meet its internal business needs, followed by the ability to meet customer demands and to analyze customer data, at 45% and 44%, respectively.
The weakest links in data architecture are lack of staff resources (53%), the length of time data is stored (45%) and the limitations of legacy systems (44%). But larger companies are far more likely to see legacy systems (51% versus 38%) and the lack of involvement by the business side in data architecture issues (41% versus 29%) as challenges, probably because of the sheer size of the organization.
Less successful companies are more often stymied by their inability to respond quickly to changing business conditions, 67% versus 52%, and their inability to meet customer demands, 48% versus 34%.
Estimates for how much the business has lost due to problems with data architecture are most likely just guesses. Still, the median amount of money lost in the past two years due to such problems for organizations with employees more satisfied with data access is $100,000, compared with $500,000 for those less satisfied. The bigger wake up call: 10% of all respondents claim to have lost more than $8 million.
Conclusion 03: Assets
In today's organization, data is both an asset and a rapidly increasing burden. With growth rates projected to go through the roof, businesses clearly need to get ahead of the curve or get buried. A common data model and a data architecture plan can help organizations make the most of their data as well as manage it, but neither is widely used yet.
Today, the median storage for companies (the midway point of all companies in our sample) is 3 terabytes, jumping to about 5.8 terabytes for larger firms. Smaller companies have less to worry about, with slightly more than 2 terabytes under management today. The averages are much larger3,408 terabytes for larger companiesbecause of the huge amounts of data some companies store.
The amount of data is projected to grow 29% in the next year at larger companies and 24% at smaller onesroughly doubling in three years.
About 57% of data at both large and small firms resides in databases; the rest is in unstructured forms such as text documents and e-mail. XML, which allows users to define how documents can be structured, accounts for a paltry 8% of data stored.
Thirty-four percent of all respondents have a common data modela consistent way to label and store informationwith another 33% saying they'll have one within a year.
Only 37% have a formal plan for creating and maintaining their data architectures, begging the question: Are companies taking the steps needed to forge a sound data architecture?
Conclusion 04: Standards & Tools
Employees are happier with data access when IT develops and follows consistent approaches to data management. Employee satisfaction also shoots up when companies encourage data usage standards, use particular data-handling tools and more effectively manage data.
Organizations whose executives say their employees are more satisfied with data access are more likely to follow companywide data management practices than those whose employees aren't as satisfiedin some cases, by substantial margins. For example, data usage standards are followed by 57% of those companies with more satisfied employees, but by only 36% of those companies with not as satisfied employees.
About 78% of companies follow standards for the database management software they use, such as Oracle or DB2, whether or not their employees are satisfied with data access. But that's where the similarity ends: Standards for tools for accessing data are followed by 75% of companies with more satisfied employees, but by only 49% of companies with less satisfied employees, and tools for generating reports are followed by 67% of companies with more satisfied employees, and just 48% of those with less satisfied employees.
Companies with employees more satisfied with data access believe they do a far better job of managing data, 74% to 16%. Good management standards clearly lead to better satisfaction.
Conclusion 05: Practices
Top IT execs demonstrate the importance of data architecture to them in part by how frequently they give it attention. They also make a statement about its importance by hiring professionals who can deal with it exclusively. But it isn't just IT that needs to focus on data architecture: By getting involved in defining business requirements that drive architectural choices, business units can have a substantial impact.
The top IT exec reviews data architecture issues quarterly or more frequently 62% of the time. But 28% of those surveyed say they look at data architecture issues once a year or less.
About 63% of our respondents say the business units at their organizations are extremely involved or involved in data architecture decisions. That number drops to 51% in organizations less satisfied with data access, versus 73% in more satisfied.
Only 44% of those surveyed keep data architecture professionals on their staffsa pretty low percentage. But they're more frequently turned to in companies with more satisfied employees, 52% to 34%, indicating that their contributions to user satisfaction with data are significant.
A solid data architecture is too closely linked to business success and employee satisfaction with data for the correlation to be mere chance. The need for a common data model and a workable plan will only increase as the amount of data stored increases and as companies seek to store more data in XML. Hiring data architects can help, but ultimately business and IT executives need to work out these issues as a team. That's the only way to make sure it can help the organization reach the goal of data transparency, so closely linked to meeting both internal and external business needs.
How the survey was done: CIO Insight designed the data architecture survey together with Advantage Business Research Inc. (www.advantageresearch.com), a Lake Success, N.Y.-based supplier of custom research services. CIOs, chief technology officers, and vice presidents of information technology and services gathered from a number of sources, including third-party lists and other Ziff Davis Media publications, were invited to participate in the study by e-mail. The questions were posted on a password-protected Web site, and 665 qualified respondents replied from April 26 to May 7, 2002. All qualified respondents described themselves as knowledgeable or very knowledgeable about their company's data architecture.