Inside eBay's Innovation Machine - 'ZIFFPAGE TITLEUnder the Hood '

Under the Hood

None of this would be possible without eBay's powerful and flexible technology infrastructure. "You can't just say you have a new piece of software that people can build their own applications on. To create this, you need the underlying architecture and flexibility," says Carey, who joined eBay at the beginning of 2006 after serving as CTO at Wal-Mart; he reports to John Donahoe, president of eBay Marketplaces, and oversees a broad portfolio including product development, operations, information systems and research labs.

The company has built a grid of interlinked computers (mostly Sun 4900 and IBM LS20 blade servers), housed in a half-dozen or so data centers, which hold several thousand boxes apiece. "To scale as we need to, we need a lot of computing power that we can turn on like tap water. Innovation is happening on small boxes, so that's where we are," says James Barrese, eBay's vice president of systems development.

The software that powers eBay's operations—and that supports third-party applications—allows interaction with any type of user application. "If you can't split it, you can't scale it," says Eric Billingsley, head of eBay Research Labs. "We've made ourselves masters of virtualization. The more horizontal you can take a system, the less costly it is to operate." When it comes to hardware and software, "it's all about splitting so you can scale individual applications separately," he says.

He uses terms such as "virtualization" and "service-oriented architecture" to mean basically the same thing: splitting up large chunks of technology, such as servers, applications, etc., to make them look like one large service. "Virtualization or SOA hides the complexity of how the services are managed and allows for increased scalability. Search is an example from eBay—it is split across multiple servers and applications, but it looks like one single service to the outside user," Billingsley says.

The split-to-scale mind-set, says Trachtenberg, has been critical to the growth of the developer network. "Large sellers used to ask to write tools that would help them work more efficiently, by tying in their own systems for managing logistics, or inventory, and so on. People were trying to do it, but eBay wasn't really set up for it." Over time, he says, "We responded to their needs, which at the same time very much synched up with our needs. We want our sellers to be more efficient, and we want to help them increase the velocity of commerce on eBay."

This whole system of hardware and software—supported by a company that can throw 500 software engineers at a single coding project—was built to adapt and grow (see "Reinvention on the Fly," page 50). That mind-set is a critical part of the eBay story.

As the Web matures, Barrese says, "the time to scale for an innovator is highly compressed. You need to find what you need quickly. It's not just us—everyone must do it if they want to work at the scale of a large, consumer Web service. It's no longer optional."

eBay has also developed an increasingly automated code-release process for its core platform, which allows it to roll out anything from a new bidding tool to recently added blogging functionality in a routine, repeatable way. "We don't have to think through every detail, like notifying the site operations team of changes to the site, or the security implications of a new method of customer interaction, because we apply a consistent framework," says Barrese. Automation also lets eBay manage its sprawling platform more cost effectively, without adding too many people.

The result: eBay is able to publish a new version of its site every two weeks, adding 100,000 lines of code, all while in use. The system is never taken off-line for upgrades or maintenance. "It's a pain point for most companies, how to update applications in a broadly distributed world, but we've gotten really good at it," says Barrese.

The company built for scale is also built for change. "I think you'll see eBay change drastically in the next several years," says Billingsley. "There will be a better experience based on how we understand what our customers are doing, a better understanding of economies, matching behaviors of buyers and sellers, building communities around commerce—social commerce is how we look at it. We have a prototype of a data-mining tool that highlights the communities that already exist within eBay without us doing anything to create them."

And who knows? The next big thing for eBay may well come from somewhere else. The flexible platform supporting some 212 million registered users is the key to eBay's future, says Carey. "As we continue to build out the core platform, the question is how we expose more pieces to our developer network," he says. "We want to continue the cycle of innovation and allow innovation to come from outside the company."