Pharmaceuticals' Data Boom

Just how much more data volume are we talking about in terms of what systems today are handling versus what they're going to be asked to be handling tomorrow?

Probably a simple thing to do would be to talk about how many individual molecules historically have been synthesized and tested by companies and how many are being synthesized and tested today, and how many will be synthesized and tested in the future.

Resource Library:

When I first entered this industry, in the early 1980s, a good-size pharmaceutical company would bring out hundreds of compounds, maybe a few thousand compounds from their drug discovery laboratories every year and test them.

Five years ago, that number—for a company that was doing extremely well—might have been 10,000. Today, companies are synthesizing and testing 10,000 in a day, hundreds of thousands a month. The issue we had then, but have now even more critically, is this: How do you find the needle in a haystack when you're dealing with those kinds of numbers?

We've gone from a scenario in which maybe you're dealing with hundreds of these sorts of research efforts and data points, most of which were in laboratory notebooks, to now being able, easily, to fill a terabyte of storage every couple of months with the amount of information that we're generating simply from our drug discovery efforts.

Data warehousing really got its start in our company primarily in the sales area because we have bought a lot of sales data from various sources, and we also generate a lot of data internally from the interactions our sales representatives have with customers and with physicians. But we are at the point now where the data warehouses that we have in place in R&D far exceed in size the storehouses we have to deal with in the sales and marketing arena. That's because of the advent of combinatorial chemistry techniques, the high throughput screening that we do, and the genomics and proteomics testing that we're doing.

There are companies out there now that are coming up with ways to use information technology to help sift through complexity.

You can't scale the people fast enough to keep up with the growing demands for information. Server management is one example. Storage management is another one that's a huge challenge for us because, again, you know, I talked about the data mining, data warehouse, data landfill analogy. We were filling mass storage at a rate that was faster than, frankly, our people could maintain it unless we were using the kinds of IT tools that we spoke about.

Where the complexity management is still very, very difficult is more around complexity of business processes and mapping of those processes, optimizing those processes, and then implementing technology to support those processes. That's where it still falls in many cases to groups of subject matter experts, in some cases with butcher paper on the walls of conference rooms to figure out what are the best processes to use. That's where we bring a lot of our own subject matter experts to bear. But yes, we're looking at those kinds of tools, but more in the back-end infrastructure area.

Another thing that we're doing that is proving very interesting is looking at a number of algorithms, especially in the research area. If we could apply them, it would have great value to us in speeding new drugs to market and also in allowing us to identify potential advantages of liabilities of some of the compounds that we want to develop. I'm talking mostly around molecular modeling, chemical simulation, biological simulation kind of work.

But the problem that we have is that even with supercomputers, many of those kind of algorithms can run for weeks if not months. What we have done recently in our drug discovery area is to begin to implement what we call grid computing, which lets us take massively parallelizable problems, break them into relatively small pieces, distribute those pieces to hundreds if not thousands of personal computers, and use idle cycles on those personal computers to do the kinds of simulations that ordinarily would have been done on a supercomputer.

If you think about it, we've got incredible amounts of CPU capacity in our PCs, yet most of the time PCs just sit there, and the fans run but they don't do much. What we've done is take a huge amount of that personal computing capacity without intruding on what people are doing at their desktops or benchtops, and we're harnessing that to tackle some of what has historically been, frankly, unsolvable problems, because of the computational load.