Is 'Big Daddy' Choking Google?By Ben Charny | Posted 05-09-2006
Is 'Big Daddy' Choking Google?
Web site operators are clamoring to understand what can best be described as an ongoing disturbance in the Google Force.
Google's search engine, once a clean, lean indexing machine, from a Webmaster's perspective has been slipping badly lately.
Starting about two months ago, site operators have complained that their Web sites have suddenly disappeared from Google's index for no reason, tantamount to disappearing from the Internet.
Another common gripe is how a Web link mysteriously drops from a prominent place in search results to Page 43 or so. The list of gripes goes on.
Something is happening, although Google hasn't said what it is yet and likely won't on its annual Press Day, which is scheduled for May 10. Google usually makes a number of impressive announcements at this event. This year, there's some "product demos" in the offing, it promises.
But in the backdrop of all the shiny new Google stuff is the 800-pound gorilla of a question: What's going on? Many people suggest that Big Daddy's to blame.
Big Daddy is, in effect, a brand-new data center that Google uses to perform core search engine tasks like cataloging Web sites or serving up localized features. Google operates thousands of data centers across the globe.
The Big Daddy project stands out because search engines infrequently upgrade the computing and networking hardware. Instead, the focus is on new features to lure more site visitors, which translates to more search engine revenues.
By upgrading, Google can better compete against rivals Yahoo, the world's leading Internet destination but second to Google in search, and Microsoft, whose online MSN unit manages the world's No. 3 search engine.The ultimate goal is a bigger slice of the $13.8 billion expected to be spent on Internet advertising in 2006, and even more in years going forward.
Every search engine is constantly tinkering, or buying new companies and absorbing their intellectual property. It's all to find new ways of attracting an ever bigger audience, which translates into higher advertising revenues.
But as all the Big Daddy disruptions seem to indicate, there's always growing pains as companies sharpen a competitive edge.
As screen-name JohnW writes in a SearchEngineWatch forum: Google "rolled out some new technology to fix some things, and in the process they seem to have broken some other things. There isn't much to say about it other than it's brokenit is what it is and will be over when it's over."
Because of Big Daddy, the theory goes, Google must completely erase the databases where it keeps its 8 billion-plus Web pages. Then it will add Big Daddy and kick-start the indexing.
Big Daddy Knocks
That explains why sites disappear, for instance, a few days or weeks at a time. It's Big Daddy knocking, say several posters at this search industry forum (password and username are required).
Another theory about the disruptions is based on comments about a Google "server crisis" from Google CEO Eric Schmidt to the New York Times. From those words comes the idea Google's plumb out of data storage. That too explains the indexing issues.
Read the full story on eWEEK.com: Is 'Big Daddy' Choking Google?