In the recent massive Visa data theft case, Visa and CardSystems officials tried hanging some of the blame on the companyCable & Wireless Securitythat conducted an audit for CardSystems long before the data loss, saying that they should have identified then some sloppy practices.
Should they have? Could they have? Security experts agree that audits are only as strong as the instructions the auditors are given.
"Some 95 percent of the organizations that I go to [for security audits] believe they are getting more than they are," said Craig Wright, the manager of computer assurance services with the BDO Chartered Accountants and Advisers firm in Sydney, Australia.
A typical example, Wright said, involves the company that hires so-called white hat hackers who get paid to try to break into systems and then report discovered flaws to that companyand sees the results of a bunch of automated tests that show no vulnerabilities. But when Wright's team looks at the system, they discover users who had full network access four years after leaving the company.
Not only are most data thefts internal, but "the people inside your network know what to take and what is the most valuable," Wright said.
The vast majority of those internal assaults are simply authorized users who exceed their authority. "One user might try transferring a lot of money to himself. 'Gee, it worked. I'm leaving,' " said Fred Cohen, the CEO of Fred Cohen & Associates and the man who is most widely credited with the earliest anti-virus efforts in the 1980s.
Audits frequently overlook another common security weak point: system interdependencies. A manufacturing plant, for example, might be wisely cut off from the rest of the network to guarantee that viruses and other problems can't get through.
But those same companies that were smart enough to isolate their manufacturing plants often link crucial financial systems with an easy to break-into domain name server, Cohen said.
Monkeying with the DNS is a very easy way to engage in identity theft and data theft because it takes traffic intended for a legitimate company and sends it to a duplicate site to grab passwords and many other pieces of information.
"DNS, everyone forgets about it. If you compromise your DNS server, you basically own the site," Wright said. "With Citibank.com, do you remember the IP address or Citibank.com?"
A smart hacker wouldn't be greedy and might redirect only a small portion of the site traffic to the dummy sitesay, perhaps one out of a thousand customersmaking it almost impossible for site or law enforcement authorities to recreate the hack, Wright said.
How can audits be used more effectively by companies? First, executives must understand what they are buying. Security audits are often confused with security assessments.
Wright points to the Cable & Wireless Security incident as a good example of the confusion. Cable and Wireless said it had "completed an audit of the systems at CardSystems. This is not correct. They have had a vulnerability assessment. A vulnerability assessment is in no way an audit," Wright said. "Vulnerability tests are generally about 13 to 15 percent as effective as an audit. Also, a well defined vulnerability test takes just as long as an audit. The level of skills required for an audit is far greater than (those needed for) a vulnerability test."
Adding to the confusion is that another well-regarded security expertMark Rasch, the former head of the U.S. Justice Department's computer crime unit and the government's prosecutor against Robert Tappen Morris, who created the first major Internet wormagrees that the terms are confusing and took the opposite position. "To me, an assessment can be more comprehensive than an audit," said Rasch, who today serves as senior VP and chief security counsel for Solutionary Inc.
Cohen comes in right down the middle, arguing that neither an audit nor an assessment is better, that they are merely different and it depends on how both are executed.
But both approaches suffer from the same problem, which they also share with their financial audit cousins: the investigators are told what to look at and what to ask by the very people they are trying to evaluate.
"There are big problems with this. Management really wants you to say good things about them," Cohen said. "They might want you to only look at the things that are problem-free. The auditors only measure what they're asked to measure."
Next Page: Finding fault with audit firms.