Vendor Value 2008: IT Providers, Running In Place

By Guy Currier  |  Posted 11-19-2008

Vendor Value 2008: IT Providers, Running In Place

Here's the good news from our annual Vendor Value survey: In a difficult business milieu, IT vendors have not lowered service levels, slowed innovation or delayed product updates. Now here's the bad news: They aren't doing much better than they did last year.

Value and reliability have stayed about the same over the past two years, says George Jackson, CIO of Triad Guaranty Insurance in Winston-Salem, N.C., who adds that "a few vendors have gotten a little more aggressive at being customer-centric with the worsening economic downturn." However, some of that energy is not well spent. "More products do not mean more solutions," points out Peace Corps CIO Edward Anderson.

Our survey results bear out these two points of view. In 2007, we saw vendor ratings gain across the board from 2006. But this year's survey shows more mixed results. From 2007 to 2008, there were 22 gainers in overall score and 15 losers. Loyalty levels, which had risen from 2006 to 2007, were stable this year.

This year's top-rated vendors performed somewhat better than last year's in providing a cost reduction or return on investment to customers. The top vendors did the best on reliability rather than value measures.

RSA Security, new to the survey this year and a surprise winner for highest overall score, got the second-best ROI rating, but the company really shone in meeting commitments on time and on budget, and in being flexible and responsive to customers' needs. Vendors that improved most from last year delivered value on customer investments, as the business climate stresses the overall value proposition from every vendor.

As in past years, security vendors scored the best and telecom vendors the worst. But the IT security market has seen rumblings of dissatisfaction, largely with the three major software vendors: Symantec, McAfee and Trend Micro. Users react to what they see as needless bundling, unnecessary added features and harder-to-use products. As a result, they turn to smaller, more focused vendors like 3Com's TippingPoint, Check Point or Cisco's IronPort. Security vendors showed the greatest drop in loyalty, compared with the other sectors.

Hardware vendors fared significantly worse this year on measures of flexibility and responsiveness, reflecting these companies' efforts to tie their product sales to larger consulting and service contracts. IT execs want this, but the implementation of these services has made them concerned about vendor lock-in.

The pressures of the tough economy may lead to a shake-out in vendor-customer relationships. "Greater industry innovation has created more products, but not necessarily better solutions," Anderson says. "This leaves the increased burden of integration, testing and documentation on the client."

The Leaderboard

Three of the most-improved vendors--Cognos, Sprint Nextel and Nortel--showed the greatest improvements in delivering either ROI or cost-cutting benefits to customers.

Value doesn't come simply from lowered costs, and reliability doesn't spring only from a long-term vendor relationship. Google's many free offerings make it a winner in the top-value category, but low acquisition investment is only half the equation. Another winner, Check Point Software, is valuable to customers for an unrivaled ability to address their business issues.

Well over half of the customers of every vendor we measured would choose to continue working with that vendor, but loyalty measures can differ from performance ratings. Companies like and Adobe seem worth staying with because of the promise their offerings represent; SAP, while not pleasing many in the past year, has plenty of residual goodwill from customers. Novell, by contrast, has had enough of an uncertain past that it clearly has ground to make up despite its rather high ratings.

Small firms working with IBM rated the company much higher than their large-business counterparts did. Big Blue may seem like an expensive choice for small businesses, but despite having to deal with relatively high up-front costs, smaller customers find benefits in the realm of cost reductions. However, small businesses tend to use IBM's pricier services offerings less frequently than bigger firms do.

Hewlett-Packard's strong relationships with its many resellers and service providers translate into reliability levels that are usually reserved for big customers, making the company a small-business winner. And Research in Motion is doing a good job addressing an area of vital importance to smaller firms today--mobility--so it's providing small businesses with much-needed solutions.

Staffers can rate vendors much higher or lower than their bosses do: IT executives are often sour on a vendor that's liked in the trenches. Though Avaya is the lowest-scoring networking vendor in the survey, the company is much more popular with lower-titled IT professionals (those with manager titles and above, but below director-level). This is especially true of its service levels. On the other hand, Sprint Nextel fared better among executives than it did among lower-level professionals.

This divide between managerial classes is a worthwhile consideration when choosing a vendor. Staffers more likely will be tasked with deployments and use of a vendor's offerings, as opposed to managing the vendor relationship. Where there's a disconnect, it often shows differing images of the vendor at the deployment level versus the strategic level. Staffers may have good reasons to prefer a vendor that managers find difficult.

Check out the next page for detailed methodology.  

How the Survey Was Done

How the Survey Was Done

What does the survey measure? CIO Insight's 2008 Vendor Value Survey measures how U.S.-based IT executives generally perceive the value of their vendors' product and service offerings, and those executives' overall satisfaction with the support these vendors provide.

How were the vendors selected? The published results include only vendors which received 49 or more qualified responses on all ratings. To create our initial list of IT vendors for the survey, we relied on the Fortune 500 and Global 500 lists and Ziff Davis Enterprise Research ongoing vendor studies in 30 technology categories.

How was the survey conducted? Ziff Davis Enterprise Research staff designed and fielded the survey, and tabulated responses. IT executives from Ziff Davis Enterprise's magazine and web site subscriber and member lists were invited to take the online survey and to speak with CIO Insight editors directly about their experiences. In addition, a smaller sampling of non-executive IT professionals was also invited; these responses were not tabulated with the main results. In all, 568 qualified respondents (257 from companies with at least $5 million, but less than $100 million in 2007 revenues--or budget if not for profit; 185 with at least $100 million but less than $1 billion; and 126 with $1 billion or more) replied and complete the survey between October 3 and October 15, 2008. Of the respondents, 204 were C-level executives at their company, and the rest held titles of director of IT or higher. Respondents were only considered qualified if they described themselves as very knowledgeable or knowledgeable about the IT vendors their company uses, and the value it has received from them.

How are vendors rated? After identifying the vendors their firm has had a business relationship within the past 12 months, and whether they use each such vendor as a hardware, software, telecommunications, consulting, or outsourcing services provider (or some combination), respondents were asked to rate those vendors as "excellent," "good," "fair," or "poor" on seven criteria. Those labeled in the results as value criteria were 1) meeting the customer's expectations for increasing revenues (or for achieving mission, if not for profit); 2) meeting the customer's expectations for lowering business or IT costs; 3) ability to solve business problems using the products or services purchased or engaged; and 4) meeting the customer's ROI or business-value expectations. The remaining reliability criteria were 5) meeting commitments to the customer on time and budget; 6) being flexible or responsive to the customer's needs; 7) meeting the customer's quality expectations for the products or services delivered. The overall rating given for each vendor is the average percent of "good" or "excellent" responses for each criterion. Additionally respondents were for each vendor their firm works with whether, if they had a choice, they would or would not continue to do business with the vendor.

Back to CIO Insight