Inside eBay's Innovation Machine

By Edward Cone

Inside eBay's Innovation Machine

Business is steady on an early december afternoon at the iSold It consignment store on Skeet Club Road in High Point, N.C., as customers drop off items to be sold on eBay. Bikes, electronics, power tools—a steady flow of stuff, forming a tributary to the nearly $50 billion flood of goods and services that will be sold on the giant online marketplace in 2006.

iSold It LLC, with almost 200 outlets from coast to coast, has become one of the largest sellers on eBay Inc. by making it simple for anyone to move merchandise across the sprawling auction site. Simple for the folks consigning the stuff, that is—they just fill out a quick form, then go home to watch their online auction and wait for a check—but a fair amount of work for iSold It's employees, who must check the items in, list them in the most advantageous areas on eBay, keep track of bidding and sales, and follow through with shipping and payment.

Multiply that by the 50,000 different auctions iSold It manages in a typical month—about 15,000 at any given moment, closer to 18,000 during the December holiday rush—and, well, "it gets very complex," says Dave Crocker, senior vice president of business development at the privately held Monrovia, Calif., company. To deal with that complexity, iSold It is switching from internally developed software to a more sophisticated application from a Salt Lake City firm called Infopia Inc., which links directly to various eBay sites and handles pricing, listing and other key tasks more efficiently.

Infopia is not just another software vendor. It's part of a growing community of some 40,000 independent developers, all building products using eBay's own application programming interfaces, or APIs—the connection points that let a program share data and respond to requests from other software. These applications are tailor-made to work seamlessly with eBay's core computing platform. eBay provides its APIs to the developers for free; its cost is limited to maintaining the code and providing some support resources for the developers.

The payoff: a network of companies creating applications that help make eBay work better, grow faster and reach a broader customer base. (eBay's other business units, Skype and PayPal, also have open APIs and developer programs.) eBay says that software created by its developer network—there are more than 3,000 actively used applications, including a configurator that allows high-volume sellers to list items more efficiently, and a program that notifies buyers of auction status via mobile phone—plays a role in 25 percent of listings on the U.S. eBay site. The company has about 105 million listed items at any given time; roughly half of its sales come from within the United States.

Sharing APIs is common practice for software companies, but eBay, along with its fellow online-retail pioneer, Amazon.com, is breaking new ground in its industry by establishing a large community of outside developers. And the implications of this strategy go much further than the world of auctions and electronic storefronts.

"It's about allowing people outside your company to write services that communicate with you-—it could be companies in your supply chain, sharing information about inventories or billing," says Adam Trachtenberg, senior manager of platform evangelism at eBay (i.e., the guy responsible for the care and feeding of the developer program).

"In the next few years you will see more and more companies with third-party developer networks, even companies you might not think of as likely candidates," adds Zeus Kerravala, senior vice president of enterprise research at Yankee Group. That could mean companies in far more traditional businesses than eBay, including manufacturers, he says. "These days, executives are telling me 'we're more like a software company than a hardware company'—and I say, then act like one. The long-term winners and losers in markets are determined by ecosystems around them, and developer networks fit that model. Developer communities allow companies to do things over the Internet with resources they don't have themselves," he says.

Companies can't just flip a switch and join the game, though, says Daniel Sholler, a research vice president at Gartner Inc. They may need to first embrace the design model of service-oriented architecture, which hides the underlying complexity of a system from users, and allows components of an IT infrastructure to be reused and recombined to support particular processes instead of dedicated tasks. "This is part of the maturation of the Web, the trend toward service-enabling all kinds of systems and sharing information more freely," Sholler says.

Companies such as Infopia are now creating mash-ups—applications built around APIs from eBay and other firms such as SalesForce.com Inc. and FedEx Corp.—that extend the relationship between different companies and their customers. The mash-ups loosely join unrelated entities, portending a new level of interactivity between companies and customers of all kinds. "This is what Web 2.0 does for business," says Infopia CEO Bjorn Espenes. "Everyone can pick and share information in different ways that are much more automated."

It's the future of commerce, perhaps, but eBay has some very traditional issues in mind: growth and competition. The company, now 11 years old, had profits of $1.08 billion on sales of $4.55 billion in 2005, but has grown so large that even the double-digit growth rates it enjoys may not satisfy investor demands. "They are in a tough spot, because they're approaching maturity with the core auction business," says Sucharita Mulpuru, a senior retail analyst with Forrester Research Inc. Meanwhile, eBay faces increasing competition from the Web operations of traditional retailers such as Wal-Mart Stores Inc. and Target Corp., and specialty vendors such as Best Buy Co.

eBay needs to expand past the online auction world it dominates. The company must help its large sellers become more efficient and grow its fixed-price marketplaces, and turbocharge its international business, which has not developed as planned; in late 2006, eBay shut down its Chinese affiliate and entered a minority partnership with a Beijing company. The developer ecosystem will play an important role in each of these areas.

ZIFFPAGE TITLEBuilding a Growth Machine

Building a Growth Machine

eBay's developer community includes venture-funded, stand-alone companies such as Terapeak (a division of Advanced Economic Research Systems Inc., based in Victoria, B.C.), which offers research tools to sellers; some in-house talent at big sellers; and individuals with bright ideas. Some of these software developers charge users—buyers and sellers on eBay—for hosted access to their applications, while others make them available for free. Most work on the sell side, where the greatest efficiencies have been gained to date.

Outside developers have brought bidding and buying tools for eBay to television, mobile and interactive-voice platforms. The PayPal online payment system, created by third-party developers around an eBay API, was so useful and popular with the user community that eBay bought it in 2002 and made it an operating unit of the company.

"All of those things are riding on top of our infrastructure, and we basically power them," says Matt Carey, eBay's chief technology officer. "They are extending our platform to places we never thought it could go. If we did things in a serialized way, with customers waiting for the next edition of our own applications to come out, it would not be anywhere near as efficient as letting 40,000 people do development." When he travels to eBay-sponsored developer events, says Carey, "it's exhilarating to see the stuff they've been working on."

Some independent developers, including Infopia, help make big sellers more efficient in eBay's established businesses. Others help push eBay into new markets, be they niches or new places. "Things don't always translate well from one country to another, for regulatory or cultural reasons," says Trachtenberg. "We work with developers to localize applications for each country site." The eBay community has developers active in almost 100 countries.

eBay started its developer program in 2000, but stepped on the gas in the last couple of years. In late 2005, it dropped fees for access to its APIs, a move that saved large developers thousands of dollars a month; the developer network doubled in size between mid-2005 and mid-2006, while the number of sellers using third-party listing applications grew by 50 percent in the same period. "The logic of making the APIs free was pretty much a leap of faith," says Carey. "A lot of momentum was building in the community, and we were listening."

Quite a change for eBay, which threatened to sue third-party developers as late as 1999. "Working with eBay has changed dramatically over the years," says Infopia's Espenes. "They have gone from allowing outside companies in, to encouraging us, and in some cases driving us to develop technologies on the platform. They've turned more to an open, Web-services architecture, and they have put a lot of resources into the people part of the equation in order to allow companies to add value on top of their infrastructure. The results are astonishing."

eBay's outreach to developers includes a certified provider program, which validates developer products and helps market them. It provides a software development kit that includes integration capabilities for the eXtensible Markup Language, Simple Object Access Protocol messaging scheme, and the JAVA and .NET software programming platforms; an online "developer zone" with access to a variety of tools and support documents; a testing environment; and a community forum for developer collaboration, as well as other member forums.

"They are way ahead of other companies," in terms of supporting developers, says one application builder who requested anonymity because he works with other e-commerce sites. "eBay's infrastructure is far superior to anyone else we work with. They've done it longer and have a much bigger community, and there is better documentation and better support."

Still, the relationship between eBay and its developers can be complicated. "They have been very helpful to us, but they also launched their own tool in our space," says Dave Frey, Terapeak's director of marketing. "They see it as choice in the marketplace, and so do we. Not every eBay seller loves eBay, so we provide something important even when they are in the same market." Sometimes, eBay likes a product so much it buys it, as in the case of a company called CARad, which makes a specialized application for listing products for sale at eBay's automotive site. "This a new wave of business," says Frey. "eBay is a supplier, a marketing channel and a competitor. It's a weird arrangement."


Under the Hood

None of this would be possible without eBay's powerful and flexible technology infrastructure. "You can't just say you have a new piece of software that people can build their own applications on. To create this, you need the underlying architecture and flexibility," says Carey, who joined eBay at the beginning of 2006 after serving as CTO at Wal-Mart; he reports to John Donahoe, president of eBay Marketplaces, and oversees a broad portfolio including product development, operations, information systems and research labs.

The company has built a grid of interlinked computers (mostly Sun 4900 and IBM LS20 blade servers), housed in a half-dozen or so data centers, which hold several thousand boxes apiece. "To scale as we need to, we need a lot of computing power that we can turn on like tap water. Innovation is happening on small boxes, so that's where we are," says James Barrese, eBay's vice president of systems development.

The software that powers eBay's operations—and that supports third-party applications—allows interaction with any type of user application. "If you can't split it, you can't scale it," says Eric Billingsley, head of eBay Research Labs. "We've made ourselves masters of virtualization. The more horizontal you can take a system, the less costly it is to operate." When it comes to hardware and software, "it's all about splitting so you can scale individual applications separately," he says.

He uses terms such as "virtualization" and "service-oriented architecture" to mean basically the same thing: splitting up large chunks of technology, such as servers, applications, etc., to make them look like one large service. "Virtualization or SOA hides the complexity of how the services are managed and allows for increased scalability. Search is an example from eBay—it is split across multiple servers and applications, but it looks like one single service to the outside user," Billingsley says.

The split-to-scale mind-set, says Trachtenberg, has been critical to the growth of the developer network. "Large sellers used to ask to write tools that would help them work more efficiently, by tying in their own systems for managing logistics, or inventory, and so on. People were trying to do it, but eBay wasn't really set up for it." Over time, he says, "We responded to their needs, which at the same time very much synched up with our needs. We want our sellers to be more efficient, and we want to help them increase the velocity of commerce on eBay."

This whole system of hardware and software—supported by a company that can throw 500 software engineers at a single coding project—was built to adapt and grow (see "Reinvention on the Fly," page 50). That mind-set is a critical part of the eBay story.

As the Web matures, Barrese says, "the time to scale for an innovator is highly compressed. You need to find what you need quickly. It's not just us—everyone must do it if they want to work at the scale of a large, consumer Web service. It's no longer optional."

eBay has also developed an increasingly automated code-release process for its core platform, which allows it to roll out anything from a new bidding tool to recently added blogging functionality in a routine, repeatable way. "We don't have to think through every detail, like notifying the site operations team of changes to the site, or the security implications of a new method of customer interaction, because we apply a consistent framework," says Barrese. Automation also lets eBay manage its sprawling platform more cost effectively, without adding too many people.

The result: eBay is able to publish a new version of its site every two weeks, adding 100,000 lines of code, all while in use. The system is never taken off-line for upgrades or maintenance. "It's a pain point for most companies, how to update applications in a broadly distributed world, but we've gotten really good at it," says Barrese.

The company built for scale is also built for change. "I think you'll see eBay change drastically in the next several years," says Billingsley. "There will be a better experience based on how we understand what our customers are doing, a better understanding of economies, matching behaviors of buyers and sellers, building communities around commerce—social commerce is how we look at it. We have a prototype of a data-mining tool that highlights the communities that already exist within eBay without us doing anything to create them."

And who knows? The next big thing for eBay may well come from somewhere else. The flexible platform supporting some 212 million registered users is the key to eBay's future, says Carey. "As we continue to build out the core platform, the question is how we expose more pieces to our developer network," he says. "We want to continue the cycle of innovation and allow innovation to come from outside the company."

This article was originally published on 12-06-2006