Inside Qualcomm's Integrated IT InfrastructureBy Peter High | Posted 02-20-2012
Inside Qualcomm's Integrated IT Infrastructure
WHO: Norm Fjeldheim, SVP and CIO, Qualcomm
WHAT: Sharing his views on Qualcomm's approach to "everything as a service" (XaaS) for its 21,000 employees worldwide
WHERE: San Diego
WHY: To provide CIOs and other IT leaders with actionable advice and insights about how to optimize the complex infrastructure of today's IT organizations
Norm Fjeldheim, SVP and CIO of Qualcomm, the $15 billion maker of communication equipment, has had an unusually long tenure as CIO, having taken on that role in early 1999.
A 24-year veteran of the company, Fjeldheim has overseen significant changes to Qualcomm's IT infrastructure. Roughly 1,400 of Qualcomm's 21,000 worldwide employees report to Fjeldheim.
San Diego-based Qualcomm, which operates 200 offices worldwide, is engaged in the development, design, manufacture and marketing of digital wireless telecommunications products and services. Its divisions include Qualcomm CDMA Technologies, Qualcomm Enterprise Services and Qualcomm Government Technologies.
In January 2012, the company was named one of Fortune's 100 Best Companies to Work For.
In this one-on-one Q&A, Fjeldheim tells CIO Insight's Peter High about Qualcomm's approach to "everything as a service" (XaaS), the rationale for choosing this approach, the challenges he met along the wayand the value he has garnered for his company.
CIO Insight: Norm, you first dipped your toe in the water on cloud computing and the SPI model [software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS)] before those terms were well known or widely used eight or nine years ago. What was the impetus for delving into virtualization?
Norm Fjeldheim: We began a little less than a decade ago when members of my team approached me and indicated that they thought virtualizing would add tremendous value to our operations. They indicated that it would render our operation more efficient, and that it would lead to significant cost savings. I asked the team what they needed to begin this journey. They said they needed a modest financial commitment to get started: a couple of hundred thousand dollars. I challenged them, saying that if virtualization was to be such a cost saver, we should expect that these activities would be self-funding after this initial seed investment.
The team began with Windows-and rather conservatively. (In retrospect, I wish I had pursued Windows and Unix together, so that it would not have looked like a Windows-specific effort in the beginning.) However, the efforts quickly built momentum. In the end, the team required significantly less than the initial commitment before these efforts became self-funding.
Our biggest foray into SaaS came by happenstance. We were a big user of Siebel for our CRM solutions. In 2005, a new business unit of ours was ramping up, and as the unit members planned the deployment of their first product, the software requirements did not include any mention of a need to ramp up Siebel in the process. As we found out about this six weeks prior to the launch of this new business unit's first product, this led to a short time frame to get the functionality up and running and therefore [ramping up Siebel became] a pressing need. We realized we were not going to meet their needs, and it led us to think about alternatives
We pursued Salesforce.com potentially as a one-off instance for us, but discovered that it had as much functionality, more flexibility, and could be implemented and ramped up in a fraction of the time of our solution. We ended up migrating to Salesforce.com completely, and saved 50 to 60 percent in costs as a result, with improved functionality.
You run IT for a very engineering-centric organization. Engineering-led organizations tend to be challenging environments for CIOs, since many people outside of IT think they can do the CIO's job better than the CIO can. Were there cultural hurdles to get over when you announced you would enact this change?
Fjeldheim: The biggest hurdle was cultural, actually. There was a general bias among engineers-in our company and beyond-to have physical equipment optimally under one's desk. We asked them to change the model rather significantly. The challenge we had in selling this was to ensure that our engineers would either get the same value at a lower cost by putting more of what they used in the cloud, or better value at the same cost. That was another challenge we as an IT department put on ourselves, and I'm happy to say that we've been able to consistently deliver against that.
What other issues did you face in those early days?
Fjeldheim: The other major issue, frankly, was that demand for cloud computing and the SPI model grew so fast that we had trouble keeping up with it. As I said, we were able to deliver against the value-to-cost challenge that we gave ourselves. Once we did, cloud computing and the SPI model took off and built a momentum of its own. That was the right problem to have in the early stages of this transformational journey, but it was a problem nevertheless. We had to play catch-up on the automation as a result.
The key at that stage was pausing and thinking about the new fundamentals we needed to have in place to ensure that we could build out the cloud, but do so responsibly. We ended up investing in management tools, monitoring tools and ITIL. We invested in processes and the automation of those processes. We used ITIL as the process framework and then bought and built tools to manage and monitor our infrastructure. As much as possible, we deployed the same processes and automation across our entire environment, virtual and physical. We went from managing 400 virtual servers to managing 4,000 of them. A tenfold increase required that we think about things significantly differently. This laid the appropriate groundwork that has sustained us through the subsequent years.
When we transitioned more of our software to the cloud, leveraging the SaaS model, we had to find solutions that would also work with our customers. In some cases, we partnered with our customers to transition together. This meant proving the benefits to our customers so that they would see this as a valuable change in their organization. We benefited by transitioning to SaaS earlier than many of our customers and had our own business cases to present as a result.
For our virtualization efforts, a lot of the tools didn't exist, so we had to build them ourselves. This slowed us down a bit and made the virtual environment much more complex. We had to keep our top engineers on the system to support it and troubleshoot problems. Later, as better tools came out, we were able to reduce that requirement, and we now have much of the system supported by more junior engineers.
For SaaS, the biggest problem was integration. The integration tools were not very good initially, so that was a big barrier to using SaaS applications, specifically SalesForce.com. SalesForce.com improved the integration tools several releases ago, and that made a huge difference. We were able to pull and push data in and out of Salesforce.com much more easily, which enabled us to use that product much more broadly [than previously possible].
IT Infrastructure: The People Side of the Equation
What was the people side of this transformation? How did you staff these efforts?
Fjeldheim: For the earliest efforts in virtualizing our infrastructure, we created a "tiger team" to lead these efforts. This team helped us develop the initial processes we would use throughout this evolution. Soon thereafter, though, we determined it was best not to separate our virtual teams and our physical teams. That has proven to be a fortunate move.
For instance, instead of having a virtual team that thought only about pushing virtual servers and was competing with a physical team that was pushing physical servers, we had one team that was evolving, based on the natural demand for and evolution of these two areas. From an early stage, this approach also helped us develop one set of management and monitoring tools for the entire infrastructure. That has helped with complexity and with security.
We went from 75 servers per one administrator to 800 [servers] per administrator now. Therefore, what we manage has grown dramatically, but our staff has not grown at the same rate. This frees up our IT employees for innovative activities and means that a larger portion of the IT team is focused on ideas that truly add value to the organization. That tends to energize people and increase job satisfaction on average across the team. Frankly, the growth in the use of various systems also absorbed a lot of [employees] who were no longer needed to work on the physical infrastructure.
After starting with Windows servers, how did you determine what to put into the cloud?
Fjeldheim: The number-one determinant was the usage profile of the infrastructure. Some engineering software can be a real CPU hog. As a result, these instances are not as clearly candidates for virtualization. Those areas that were not CPU hogs were put in the plan for virtualization.
In the first five years, 80 percent of what could be in our private cloud was put in our private cloud. This includes all of our applications running on Linux and Windows servers, but does not include engineering tools [such as] Synopsis and Cadence running on our engineering compute grid, or applications running on big Unix servers [such as] Oracle applications. It took another two to three years to virtualize the last chunk of infrastructure.
Now that it's been nearly a decade since you embarked on this journey, what are the primary advantages to cloud and the SPI model based on your experience?
Fjeldheim: Our first concern is meeting the needs of our business. Demand for IT services always seems to be increasing, and the patience of the business is decreasing. This presses IT to operate with agility. My colleagues outside of IT are some of the savviest technologists around. If I can't meet their needs, they'll find other ways to be sure their needs are met. I need to be sure that we are always the first choice when it comes to technology development. We have been in growth mode as a business for a while, and I'm proud of how IT has supported that growth. The more often we can provide value-enhancing technology, whether it is new software or the infrastructure backbone, the better we will serve the needs of our customers ultimately. Cloud computing and the SPI model are a big part of allowing us to get there.
We have found that the biggest benefit for us was speed. We can deploy a virtual server in less than 15 minutes, where it used to take up to six weeks to put a new physical server in place. It used to take days to get a server up and running, and it would come out of the other departments' budgets. Now it takes five minutes, and the cost is in my budget. Needless to say, costs are dramatically lower.
The cost savings of this approach can't be overstated. We believe we have generated $25 million to $30 million in savings in hardware and power. We have also significantly improved our service levels, as unplanned downtime dropped by almost 90 percent when going from physical to virtual servers. Moreover, this allows us to render variable a lot of what were once fixed costs.
We can ramp up our infrastructure, for instance, at the same rate that demand for the different facets that make up our infrastructure is growing, rather than having to purchase significantly higher capacity than what we need. For other companies that are unfortunately in cost-cutting mode, [the lesson is that] this also allows them to make cuts in infrastructure before having to cut the firm's most valuable resource: its people.
What advice would you offer to CIOs who are at the beginning of the journey on which you are now so far along? If a CIO is considering the cloud and the SPI model, how would you counsel him or her to plan and enact this transformation?
Fjeldheim: My first piece of advice would be to "go big" on these changes because we have found the deeper we got into cloud computing and the SPI model, the more value was created. I know a lot of CIOs want to pilot this and bite off the various aspects in very small chunks. I know that many of my peers have had an experience similar to mine when they pushed aggressively for these changes.
Second, plan for success. Think about the people and processes to guide the process, and ensure that you can ramp up quickly.
Another people point is to help IT and the rest of the organization understand the value they can get from this sort of transformation. These days, companies have many examples of successful transitions such as the one I have described here. This is an advantage I would have liked to have had when we embarked on this journey.
It is also important to think of the human element in this equation. Retrain your staff so they can adequately support the growth in cloud computing and the SPI model.