Methodology: How The Survey Was Done

By CIOinsight  |  Posted 12-04-2006 Print


EUC with HCI: Why It Matters

What does the survey measure? CIO Insight's 2005 Vendor Value Survey measures how U.S.-based IT executives generally perceive the value of their vendors' product and service offerings, and those executives' overall satisfaction with the support these vendors provide.

How were the vendors selected? The published results include only vendors which received 45 or more qualified responses on all ratings, and were used by at least 70 respondents. To create our list of the most widely used computer hardware, software, telecommunications, IT consulting and outsourcing companies in the U.S., we relied on several sources for revenue and market share data: Baseline magazine's 50 Fastest Growing Software Companies, Gartner (which provided CIO Insight with lists of market share leaders), the Fortune 500 and Global 500 lists, the ForbesFastest Growing Technology Companies list, the CBR 50 IT Services Vendors list, Hoover's Online, and annual and financial reports of individual companies. Most of the companies that were included in the 2005 Vendor Value survey were included this year's survey; usually deletions were due to insufficient or unqualified responses. One company, CDW, was included because it was suggested by many respondents of the 2005 survey. Our aim was to provide readers with information on 40 of the most widely used vendors in these categories, out of 56 that were included in our survey. We limited hardware companies to manufacturers of PCs and servers, storage equipment, and networking equipment; and software companies to producers that create and sell software intended primarily for business use across many industries. We included companies that provide consulting, systems integration, outsourcing and IT services (other than data processing services and business process outsourcing services) but do not derive most of their IT service revenues from the defense sector, information processing services or customers outside the U.S. This year's survey also includes telecommunications service vendors, including companies that solely provide wireless services.

How was the survey conducted? CIO Insighteditors designed the 2006 Vendor Value and Satisfaction Survey together with Equation Research, LLC (www.equationresearch.com), an Estes Park, Colo.-based supplier of custom research services. IT executives gathered from Ziff Davis Media publication lists were invited to participate in the study by e-mail. The questions were posted on a password-protected Web site, and 826 qualified respondents (350 from companies with between $5 million and $99 million in revenues, 248 between $100 and $999 million in revenues, and 228 from companies with over $1 billion in revenues) replied from September 19 to October 8, 2006. Of the respondents, 51 percent were the top IT executives at their company, and the rest held titles of director of IT or higher. Respondents were only considered qualified if they described themselves as very knowledgeable or knowledgeable about the IT vendors and consultants their company uses, and the value it has received from them.

How are vendors rated?After identifying the vendors they have had a business relationship with in the past 12 months, and whether they use the vendor as a hardware, software, telecommunications, consulting or outsourcing services provider, respondents were asked to rate vendors as "excellent," "good," "fair" and "poor" on seven key criteria. Four of the criteria concerned value: 1) how well they have met their company's expectations for increasing revenues (or achieving mission, if not-for-profit), and 2) for lowering business or IT costs; 3) how well they have solved the business problem their products or services were purchased or engaged to solve, and 4) have met their company's ROI (business value) expectations. The other three criteria focus on reliability: 5) how well they have met commitments to their company on time and budget 6) how flexible and responsive they have been to their company's needs and 7) how well they have met their company's quality expectations for their products and services. The "overall" rating is the mean of respondents answering "excellent" or "good" for these seven criteria. In addition, respondents were asked whether, if they had a choice, they would or would not continue to do business with each individual vendor. Unless otherwise noted, percentages given are the percentage of respondents who answered either "excellent" or "good."

The exact wording of the survey questions is as follows:

Please indicate which of the following vendors your company has had or continues to have an ongoing business relationship with during the last twelve months.

Please indicate the nature of the relationship your company has (or had) with ______ in the last 12 months.

  • We have used them as consultants on business and IT strategy and/or internal IT projects
  • We have used their services to outsource IT systems and projects
  • We have used their telecommunications services.
  • We have used their hardware products
  • We have used their software products
  • We have used their security products and services.

    Please rate _____ on how well their products and services help your company achieve the following goals and expectations:

  • Meeting my company's expectations for increasing revenues (or achieving mission, if not-for-profit)
  • Meeting my company's expectations for lowering business or IT costs
  • Solving the business problem their products or services were purchased or engaged to solve
  • Meeting my company's ROI (business value) expectations
  • Meeting commitments to my company on time and budget
  • Being flexible and responsive to my company's needs
  • Meeting my company's quality expectations for their products and services

    If you had a choice, would you continue to do business with ______?


Submit a Comment

Loading Comments...
eWeek eWeek

Have the latest technology news and resources emailed to you everyday.

By submitting your information, you agree that cioinsight.com may send you cioinsight offers via email, phone and text message, as well as email offers about other products and services that cioinsight believes may be of interest to you. cioinsight will process your information in accordance with the Quinstreet Privacy Policy.

Click for a full list of Newsletterssubmit