One CTOâs quest for answers about building large systems with low-cost commodity IT.
For the past several months I've been exploring the concept of cloud computing. There has been a good deal of interest in it from the technology and business press, mostly based on what Google and Amazon.com have done to create very large pools of low-cost computing, connectivity and storage resources on which they run the major parts of their businesses.
There are attractive aspects to the approach, especially in terms of rapid and cost-effective scalability and the ability to build large business automation systems out of low-cost commodity technologies. We can learn from Google's and Amazon's experiences, but does the idea apply to mainstream business computing?
I don't have firm conclusions yet, but here's my thinking so far. As usual, the devil is in the implementation.
Let's start by distinguishing the cloud concept from software as a service. I'm not looking at shifting all my production code to software-based business services from third parties. Instead, I want to replace the multiple and distinct vendor-specific computing, network and storage products that make up my on-premise production environment with my own clouds of computation, connectivity and data storage.
Then, when I develop applications or business services that require these resources, I won't need to specify what the operating system or physical network or data storage mechanism looks like. I'll just specify how much of each kind of resource I need, and the cloud will figure out how to get it for me.
This potentially simplifies the range of skills I need in development, support and operations, although the skills themselves will need to change a lot. Today, my developers write code that is directed at how a specific product--say, a directory manager--works rather than at a capability they need--in this case, a directory service. That locks application code to products, and all too often products are locked to specific software and hardware environments. This requires more specialized developers and a more complex production environment than I prefer, making it hard to deliver high availability and pushing up total cost of ownership.
I can break some lockstep issues with virtualization, but I'm still tightly tied to specific hardware and software environments. What's more, not all workloads that might adapt well to the cloud environment virtualize well, often because of communications and data I/O constraints.
The cloud idea is good but the technology isn't quite there yet. I'd like to see more well-developed and more open standards focused on how to architect a cloud, and more products designed for cloud use -- or that can be made cloud aware. I'd have to do a lot of low-level engineering for myself, rather than getting what I need from my technology partners, because these engineering skills are rare. That adds to the initial development risks and to the longer-term support costs, especially if my home-engineered solutions don't match whatever standards emerge.
None of this is a deal breaker if you have the right engineering expertise.
This has all the signs of an immature but rapidly evolving set of ideas that must be hardened for widespread business use. My plans involve building a small generalized cloud (we already have some small specialized clouds in production) in the next few months and using it to determine what we'll need to do to develop code that runs in this environment. We plan to examine all our workloads to see what characteristics point to the cloud as the right platform.
I'll let you know what happens.