Migration CostsBy CIOinsight | Posted 06-14-2002
Roundtable: Open Source Goes Mainstream
You need one more decision on your desk, right? One more of those CIO issues that go beyond technology into the messy realms of people and politics and philosophy. Well, it will be hard to escape this one: open source softwarein particular, the operating system known as Linux. Created by Helsinki University student Linus Torvalds in 1991, and developed and still maintained over the Internet by thousands of hackers worldwide, Linux is now said to have the fastest market growth rate of any operating system in the world. Moreover, chances are good that even if you've never officially signed off on Linux, someone in your shop is experimenting with it right under your nose.
To get a better idea of what this means for CIOs, CIO Insight Deputy Editor Terry A. Kirkpatrick recently convened a roundtable of nine experts to chat about itcorporate IT execs who have deployed Linux, analysts who have studied it, and vendors of both open and proprietary software.
If there was consensus in this diverse group, it was this: Open source software should be treated no differently than any other softwarethe migration, testing and support issues will look familiar. These are the "hidden" costs that could wipe out any cost-savings on price. At the same time, however, there may be unexpected savings on hardware.
Beyond the technical issues, though, there are other considerations. Linux is both an operating system and a development model, "a dessert topping and a floor wax," as one roundtable participant put it. Open source comes "shrinkwrapped" in a philosophy of software creation and ownership that is at sharp odds with the prevailing proprietary model, and this continues to stoke a shrill debate between the two camps. Should these "soft" issues matter to a CIO faced with hard business decisions? The discussion began with the issue of costthe price of open source software being its most notable feature and something that tends to catch the eye of CIOs in these lean times.
CIO Insight: Do CIOs really care that software is free, as long as they have a tool that solves some business problem?
QUANDT: The fact that Linux is available at a low cost is only one factor, obviously one that can help the company have a return on investment, but the capabilities and reliability and performance of Linux are really much more important.
CAREY: The cost of software plays a small role at Merrill Lynch. We look for other value propositions, whether it's open source or not. Linux is just a product to us with a cost that we look at over its total life cycle.
George, you've written for Gartner that in more complex deployments, the price advantage of open source can disappear. Can you elaborate?
WEISS: As we move up the curve of more and more complex applications, and face the issues of integrating legacy environments, larger performance and database scaling requirements, these will require more systems integration, performance tuning and validation of the software. And that's going to become much of the cost of the deployment of these systems. So the initial cost is really minor.
ROBBINS: IT budgets have been dramatically affected in the last 12 months. Some institutions are moving to Linux not because the software is cheaper, but because it runs gracefully on cheaper boxes. It's very compelling to those management teams who don't really care so much about the community-based principles or the intellectual property issues. So you're seeing certain doors open to Linux, driven purely by budget.
Steve, you have just made a major deployment of Linux at Credit Suisse First Boston that I understand to be mission-critical. To what extent was cost of software a factor?
YATKO: Actually, it's the crown jewel of our firm, a mission-critical, global order management architecture. This system deals with roughly 35 million transactions per day around the world. We were looking at a significant increase in volume, driven by the way the business was shifting its trading strategies. Our resources were giving us very little additional bandwidth to grow, so we almost had to go with Linux. But it had always been not so much Linux but the operating environment that it would be installed on. We needed stability and availability as much as we did performance. So it was our choice of hardware, actually. We had investigated Egenera Inc.'s solution and felt that it could stand up to any other. After running for a couple of months on the hardware platform, we recognized that this was an incredible performance increase, in some ways 20 times the performance over traditional RISC. So we went from a four-way RISC box to a two-way Intel-based system for the hardware package.
On the other hand, Mike, you looked at Linux for Royal Caribbean at one point, and you were hesitant.
SUTTEN: Yes, Royal Caribbean has a lot of older, peculiar systems, because it's very hard to go out and retrofit a ship. So the guideline we've been using on all open source, and not just Linux, is that if it's higher up in the architecture, at the logic layer or the presentation layer, and the life of the product is somewhat limited to three to five years, we were pretty happy using open source. We think it's sustainable; it'll last for that time. But as we move further down, to the data layer, the operating system layers, we've been pretty hesitant, because we expect 10 to 20 years out of that level in the application. And so we've been hesitant going in that direction.
Photographs by Thom O'Connor (Rick Carey, Paul Cormier, Jonathan Eunice, Jason Matusow, Stuart Robbins, George Weiss, Steve Yatko), Jeffrey Salter (Mike Sutten), and Chuck Nacke (Stacey Quandt).
That raises the question of migration costs. What's involved in migrating?
EUNICE: A lot of the costs are strategicmigration, training and risk. People from the Linux advocacy community don't often think about the amount of risk and difficulty. But if you have a staff of tens, hundreds, sometimes thousands of people who know the solutions, and you already have experience with a system, whether it's Oracle or IBM or Sun or whatever, it really doesn't matter what the solution is. Choosing anything different brings up huge risk costs and migration costs that are not just migration of a particular application but also migration of who you are and what your strategy is. So you have to be pretty concerned with those. Often in the open source debate, we kind of walk around those risk and integration issues, but that's at the core of what a CIO or CTO really needs to be thinking about, these big strategic costs.
ROBBINS: I recently had a conversation with a number of CIOs around the Oracle/DB2 debate, and they have all said that they would have to be dragged kicking and screaming to do a database migration, regardless of the performance of the newcomer on the block, simply because migrations of anything at that level are always painful. It's like exploratory surgery. Until you get into the systemwhich you probably inherited from your predecessor or from a team of people who didn't document their code and have left your companyyou don't know what the real implications of that migration are going to be until you're already in it.
QUANDT: One migration issue is that Linux is moving beyond the entry-level infrastructure to the mid-range, and you're talking about databases, but there will be ways to scale Linux up to an eight-way SMP on Intel. So the characteristics of which application you can actually support with Linux are changing.
CORMIER: I agree 100 percentLinux is moving into the more complex and higher-end technologies daily. One of the things I wanted to comment onI didn't get a chance to jump inwas the question of whether Linux or open source in general is going to be around for 10 years. That's one of the beauties of Linux. You, as a customer, have the decision of how long it's going to be around for your application. You're not locked in, you're not hit with a vendor saying, "We're going to retire this next week."
YATKO: You have to ask that question with all your operating environments. Right?
YATKO: I think Linux is here to stay. I think there's enough movement going on in the industry, enough support now from major contributors such as IBM investing billions of dollars into this, and now you see Sun jumping in as well.
MATUSOW: As you talk about Linux moving into the higher end, from the CIO perspective, one of the things that you have to consider is how much test and development burden you are taking on versus what you were traditionally looking to vendors to do. There are so many different distributions of Linux out there, and they lack binary compatibility across them. If you are having multiple implementations, particularly if you have IT staff that is doing it without your knowledgewhich certainly we have found in our discussions with customers to be the casethen the modular nature of the system, in fact, is such that there is no full regression test pass being done. So that test burden is being moved onto the shoulders of the customer.
EUNICE: That's a burden that is, unfortunately, moved onto the shoulders of the user with any new coming-up technology. We've seen it with Unix in the late 1980s, we saw it with Windows in the 1990s, and we're seeing it again with Linux.
QUANDT: If you support your own application, yes, you probably have to do some of your own testing, unless you partner with a Linux distribution provider. But a lot of the leading Linux distribution providers have certified a distribution with key independent software vendors.
MATUSOW: If you talk to the customers who have joined us here today, large enterprise organizations, they all have custom applications with significant development staff. So there's a limit to how far they're going to be able to move from the traditional point of view where they're going to have to test those pieces. But if they are also adding them on top of all fixed testing and future compatibility and binary compatibility testing, it can be a very significant amount of work.
CORMIER: In terms of testing, the open source model actually lends itself to a great degree of testing just because of the nature of the model, the number of eyes that are on it. It opens itself to a great deal of generic testing, which any vendor also has to worry about. I contend that custom applications are no different from one operating system to another on where the testing burden lies. And in the case of commercial applications from independent software vendors, we are just now seeing them, as Stacey says, starting to certify to various Linux distributions. So I don't really see a problem here compared with other operating systems.
Open source is a movement as much as a softwaredoes that help or hinder it?
WEISS: There's a philosophy and a methodology that believes in the strength of community and spreading the collaborative effort, at least in terms of the number of people who might be available worldwide to look at the code and siphon out any problems, as opposed to total dependency on one vendor's organization and whether they're going to be able to uncover all the problems that a user might have.
On the other hand, there's a lot of software out there that has a great deal of intellectual content, has a lot of creative aspects to it. One would have to question whether the open source community has the creativity, the interest, the excitement or even the knowledge of where to target its efforts. So you would have to ask: Does the open source community know what the market needs and requires? When you get into the more creative aspects of delivering ROI to specific application environments and for IT organizations looking for the software developers to be able to come forth with their needs in mind, I would probably have less confidence in an open source model.
ROBBINS: In a knowledge-based economy, any approach that tends to aggregate value rather than deal with value in a cylindrical or hierarchical fashion will model the way the world is going. Over the course of time, we're moving to a decentralized, intellectual property-based economy. And any applications constructed to mirror that approach, at whatever level they operate, whether it's matrix organizations or operating systems that are decentralized, you're going to see adoption, because it's mirroring an economic trend.
How do you separate the philosophical issues surrounding open source from the hard business decisions?
CAREY: Merrill Lynch is an early adopter of Linux, and we're going to be fairly aggressive with it. I think we do it not so much because of a belief that open source is the right way. I don't believe there's any right way. We're moderates. We believe it's a good way to do some things, just like the proprietary model is a good way to do some things.
Merrill is obviously a huge user of Microsoft technology. We like the fact that Linux and Microsoft run on similar hardware. That helps us because it gives us choice, and when we have choices, we can make better business decisions. So it's not a religion, it's about business.
Will the IT shop of the future be a mixture of open source and proprietary?
CAREY: It will be at Merrill Lynch.
YATKO: It will be at CSFB.
ROBBINS: If that's true at Merrill Lynch and CSFB, it bodes well for higher adoption rates in smaller and mid-size companies, because your boards of directors won't see you as taking a risk, the way five years ago they would have. Five years ago, open source shareware was considered a radical, almost revolutionary approach to IT, and you had to do it in the background, you had to do it in private. And once it was all done, you might ask for forgiveness. Now you're seeing adoption. I have a number of clients who would feel relieved to know that Merrill Lynch and CSFB have done some due diligence here.
MATUSOW: You take an organization that is, let's say, half the size of Merrill Lynch. Do they have the capacity to do what you guys do with even Windows or with any other platform? The capability that you are making use of for an organization of your size, with the engineering capacity you have, is a luxury that I would say the vast majority of other customers do not have.
CAREY: I wouldn't call it a luxury; I would call it a necessity.
MATUSOW: A necessity is fine, but it's still a capacity that very few organizations have.
EUNICE: Every place I go has some requirement to do that kind of qualification and testing, even in a smaller or medium enterprise. It would be wonderful from a user point of view if you could order it up from a genie. That doesn't happen. The people who run IT at any company are the ones responsible for seeing that it is available, that it works right. And the onus is always on them.
MATUSOW: But there's a difference between configuration concerns for each customer, and each and every customer does have those concerns, and the process of doing their own build of an operating system.
EUNICE: I haven't seen in the Linux community everyone doing their own build.
YATKO: From CSFB's perspective, we're very similar to what Merrill is doing. We have a global build for every operating system we have. It gets scrutinized on every installation, every patch that gets installed. Linux is just another strategic operating system that fits into that model.
WEISS: Can I ask you, though, since Linux is really a set of operating systems and releases, I'm just wondering how you will be treating the various releases and versions, and whether you would standardize on one distribution's operating system, and whether you are repeating some past examples of lock-in?
CAREY: We are not going to standardize on any one distribution. In fact, we've said very aggressively LSB (Linux Standard Base) compliance is where we're going to be pushing people toward. And once we have LSB compliance, I think that's the baseline for us to pick and choose, and we may have three different distributionsone for the mainframe, one for the mid-range server and one for the desktop. And then we modify those to make them most effective in those three ranges.
I want to shift to support for a minute. Paul, if I make a bet on Linux, do I also have to make a bet on the long-term viability of Red Hat?
CORMIER: Is that a question?
CORMIER: If you make a bet on Red Hat Linux, you will obviously have to make a bet on the long-term viability of Red Hat, but not to the extent you would with proprietary software. In my opinion, Red Hat is in the best position to give the best support for our operating system. But down the road, if another vendor could do a better a job at supporting Red Hat Linux, they have the source code available to do that.
Jason, is Microsoft's shared source initiative an acknowledgement that in supporting software, the open path is better? That giving customers a peek at the code helps them fix bugs more quickly?
MATUSOW: First off, there are many different models that make up the software ecosystem. And so to say that any one model is dominant over another would be a mistake. I think we've learned from open source, particularly in the nature of community and how we can better work with customers, and so the shared source program we put forward is a way for any software vendor to be able to share source codes and still own its intellectual property. We're working on customer and partner benefits rather than dealing with the philosophical battle of whether or not you are "open." So we've been sharing source code with enterprise customers, systems integrators, governments and academic institutions, and that's just for the Windows code base. We're certainly taking lessons from the open source community as it relates to working with our customers, but we are most certainly not open-sourcing Windows.
ROBBINS: There are a couple of old CIO curmudgeons who are likely to resist anything new. Their notion of support is not so much vendor-oriented as it is in-house expert-oriented. With open source, there needs to be an individual or a team that's fluent with and dedicated 24x7 to the support of your open source platform and application. Up until maybe 12 months ago, that was not seen as a risk factor for most of these CIOs, because their staffs like new technology. And you want to keep your staff motivated. However, they've all lost a lot of head count, including expertise in this area. And so it's reminded them that when you have in-house dependencies and you hit a recessionary cycle, you could be forced to trim staff and lose the person providing the support for this platform that you have customized wildly out of necessity.
YATKO: In some ways I totally agree with you, but if this is a strategic operating system for your enterprise, a big mistake would be to put very few people on it. You have to have teams of people that understand the operating system. You have to treat it as you would any other operating system. The risks of this operating system over any other should be exactly the same, and you need to treat it as such.
MATUSOW: One of the interesting things we've heard from our customers as we've gone out sharing source code was a strong desire for us not to share the source directly with them but rather with system integrators, because they would prefer to work with those companies and allow those companies to have that intimate knowledge rather than bringing it in-house.
YATKO: When you start looking at staff being reduced, you start wondering about server consolidation. And, again, when you're getting those price performance benefits, you can start reducing any number of RISC boxes into these more commoditized hardware solutions that need only one person instead of 10. So when you're reducing staff, this is an even more important solution to look at.
If Linux is so great, why isn't everybody using it?
WEISS: It could be that we're seeing bottom-up creep in organizationsit's there, but we don't know the extent exactly. But what I think is happening is that Linux is on a curve, a growth curve. It started from a point like zero, and that was maybe 10 years ago. So the question of whether it should be pervasive is more an issue of maturity and acceptance and adoption. I don't look for massive changes in IT infrastructures. I see more in terms of trends in adoption and whether it is displacing other systems. So I would be more interested in knowing whether RISC is being displaced by Linux, for the reasons that Steve mentioned, about performance and dramatic changes in scalability. Is Windows being displaced and, if so, where would it be displaced most?
MATUSOW: George, would you categorize that displacement as a Unix displacement or a Windows displacement today?
WEISS: I would say that most of the interest I come across is being able to consolidate highly distributed and dispersed Unix boxes. However, I also believe that every placement of Linux represents a loss for Microsoft and Windows at the same time. Where Unix isn't able to really fulfill, on price performance in the mid-range and lower end, Microsoft with Windows was scaling up and intending to take a large part of that market. But the more market that Linux takes, the less Microsoft has.
CAREY: I'm going to disagree with that. If Linux is going to displace anything at Merrill Lynch, it will displace Unix. There's the real push, and I think if that actually pushes open a door, then both Linux and Microsoft have a much better opportunity. And for us at Merrill Lynch, it's a better risk profile because we like the competition. If Windows is better, then we'll run Windows. And if Linux is better, we'll run Linux. I don't have to go buy hardware again. I don't have to go and negotiate with yet another Unix provider.
So competition is good?
CAREY: I think it's great for Microsoft. I think Jason can't say it out loud, but I think he really loves Linux and he just doesn't want to say it.
WEISS: What I meant about Microsoft, and you may disagree with this, but from what I understand from users who have very highly integrated NT and Windows deployments in their corporate enterprise with lots of skills and support and Microsoft applications, is that it is exceedingly difficult for them to consider Linux in any more than maybe point solutions within that network. To try to uproot the entire Microsoft infrastructure that many users have makes it a bigger migration problem than I think they feel with Unix.
QUANDT: I'm also hearing a lot from companies about migration away from Microsoft to Linux. Companies that have already deployed Active Directories are asking about the return on investment of Linux and thinking, well, maybe they've made a mistake. Maybe they should really consider Linux now.
WEISS: Yes, Stacey, but it's in hindsight
QUANDT: I agree with you, it's hindsight, but they are actually actively doing requests for proposals.
Where do you see open source in three years, five years?
QUANDT: Open source is going to continue to move further into the enterprise, and Linux will become the dominant Unix operating system within the next three years. I think that it's going to change the way that companies think about technology.
CORMIER: Open source is proving itself to be very viable, and I think you'll see more projects in open source. You'll see Linux move higher up into the enterprise, as well as move into specialized operating systems where you may have seen RISC-based systems in the past.
SUTTEN: At the higher ends, we're going to be sharing source code over the Internet, and I think that's going to be pervasive. Until the support and sustainability issues can be solved, I don't think Linux is going to be a major enterprise player at the lower levels.
MATUSOW: Open source is going to continue to be an important part of the software ecosystem. I think that for all independent software vendors, of which I would include Microsoft in terms of being a commercial software vendor, a very healthy process is under way in which we're looking at source code access and what it means for customers and partners, and I think that trend will continue. I don't believe that by definition that means open source software will become the dominant factor in the industry. I just think it'll be a player.
WEISS: Open source development models will be increasingly accepted by software vendors and consumers, but will not replace proprietary licenses as the primary commercial software licensing model through 2006. And probably 80 percent to 90 percent of IT organizations of 100 or more employees will have some open source in their organization.
EUNICE: I like to make the distinction between open source as a development methodology and as a business model, and I think we've often thought of it as bothdessert topping and floor wax. And I think that open source is proving itself successful as a development methodology. It has had a lot of trouble as a business model.
ROBBINS: I agree. The interest lies in the fact that the software is a mirror of something larger. We have a real struggle with the notion of intellectual property. And we haven't wrestled with the whole notion of what is proprietary versus what is owned by the community. And so open source software will drive us closer as businesspeople to confronting that larger issue, and there will be some institutions that can embrace it, and there will be some that absolutely cannot.
YATKO: I agree with Stacey's points. We're seeing open source and Linux being adopted by the larger corporations such as IBM and Sun. As the ISVs are now beginning to adopt it, it will certainly start penetrating the enterprise at a much bigger level. And we'll probably see it start accelerating, since people cannot, in these economic times, ignore the cost benefits.
To CIOs considering open source, what would you suggest they think about?
CAREY: It gives choice, and when you have choice, you have more efficient markets, and consumers get the best value for their dollar.
WEISS: Users have to be cautious that they don't get too lulled into the fact that a vendor will raise the banner of open source and then, in a backhanded fashion, continue to promote its own proprietary software and try to lock the user in.
MATUSOW: CIOs need to keep in mind that there are both benefits and risks, and that clearly as a development model there are benefits. But in terms of the business model, there are risks. They need to be aware of the testing. If I'm in a regulated industry, what do I need to be aware of in terms of what my procurement processes are requiring, and whether or not my open source vendor is going to be able to tell me whether certain security standards have been met?
EUNICE: Our industry has been converging for 20 or 30 years, and trying to standardize and come to common platforms. So I see Linux as yet another one of the major pieces, certainly not the only piece. But it's a way of consolidating what we do across a lot of different hardware and processors. And it's an ugly process. I mean, Unix was, Windows wasthe whole process is ugly, but the place we want to get to is that kind of approved common platform.
ROBBINS: Another thing the CIO has to always consider is how this decision affects the company's core mission, and to make those cost/risk decisions based upon that. If you're not into sharing, and you don't believe in leveraging knowledge, and you don't believe in leveraging the benefit of others, don't do open source. There's no reason to. If, however, your business is cross-functional and non-hierarchical, you'd better think about this.
Please send comments on this story to email@example.com.
Web Sites and Papers
M icrosoft's Shared Source Initiative
Red Hat Inc.
Jonathan Eunice's essay, "Linux Ready for Prime Time"
Articles by Stacey Quandt are on the Giga Information Group Inc. site
Articles by George Weiss are on the Gartner Inc. site