Article

Published: June 2008

Introduction

Microsoft IT is facing an issue that affects many organizations. Data centers are reaching space and power-consumption capacity rapidly, while many servers that data centers house are running at very low utilization. To address these issues, Microsoft IT is pursuing virtualization aggressively as a way to reduce the number of physical servers running in the data centers, while still providing the same services to Microsoft business groups. However, implementing virtualization has meant that personnel within many Microsoft business groups have had to change their thinking regarding server resources. Microsoft is using several approaches to drive virtualization.

Identifying the Issue

For several years, the number of servers in Microsoft IT's primary data centers grew rapidly, while utilization of many of those servers was very low. Microsoft has about 13,000 servers that provide computing power to 550 buildings in 98 countries. As early as September 2005, Microsoft IT calculated that the average CPU utilization for servers in data centers and managed lab environments was less than 10 percent and dropping.

At the same time, many business groups were purchasing more-powerful servers to replace underutilized end-of-life servers. For example, some business groups replaced servers that were reaching the end of their hardware life cycle and running at less than 5 percent utilization with servers that were six times as powerful, without increasing the new server's workload.

Microsoft IT also did not have a consistent capacity-planning mechanism within business groups and company wide. In 2005, approximately 40 business groups could purchase servers and place them in corporate data centers. Many of these business groups did not have a consistent planning process for purchasing hardware. Additionally, each business unit operated autonomously, with no planning for hardware capacity across business groups.

In early 2007, this issue became acute, because the Redmond, Washington, data centers rapidly approached their capacity for space and power. Although Microsoft IT implemented the RightSizing initiative to encourage business groups to move to virtualization, and virtualized more than 1,000 servers, business groups continued to prefer buying physical servers rather than virtual machines. As a result, the purchase of physical servers was increasing at a rate much faster than was the number of virtual machines.

By May 2007, the Redmond data centers essentially reached capacity. Microsoft IT adopted a one-in, one-out approach to server deployment. If a business unit wants to deploy a server, it must identify another server to remove from the data center, which means that deploying a single server in the data center can take months.

Identifying the Solution

Microsoft IT identified server virtualization as a prime strategy to address server underutilization and data-center capacity. By implementing virtualization on the Windows Server® 2003 operating system with Microsoft® Virtual Server 2005, Microsoft IT could run multiple virtual machines on a single physical server. Because many of the servers, particularly in the development and test-lab environment, ran at very low server utilization, Microsoft IT could run an average of 25 virtual machines on one physical server in a lab environment and as many as 10 virtual machines on one physical server in a production environment.

This solution offers very real benefits:

• Deploying virtual machines rather than physical computers decreases the total number of servers in the data centers.
• Deploying multiple virtual machines (each replacing a physical server of two to four rack units) on one physical server of four rack units significantly decreases the space used in the data center.
• Implementing virtual machines significantly decreases the power required to run the servers. On average, a virtual machine with average utilization requires about 90 percent less power than does a physical server.

Implementing the Solution

Despite the benefits of deploying virtual machines, Microsoft business groups have not been quick to adopt the new technology for several potential reasons:

• Business groups typically spend freely on hardware purchases and hosting costs. With the success of Microsoft as a business, most business groups have the money available to purchase new equipment regularly.
• Business groups feel a sense of ownership of the physical server assets, so business groups perceive the move to a model of sharing assets as a loss of control.
• Business groups want to replace existing servers with similar hardware. For example, if a business group has been running its applications on a mid-size server, it typically is most comfortable with replacing an older mid-size server with a new equivalent server that has significantly more computing power.

To encourage business groups to adopt virtualization, Microsoft IT is using two interconnected approaches:

• The Compute Utility strategy and RightSizing initiative. Begun in the summer of 2005, these efforts focus on developing a compute-utility concept and identifying the servers that are good virtualization candidates. The Compute Utility strategy started the process of implementing virtualization and has used a variety of tools to encourage business groups to adopt virtualization. The RightSizing initiative. one of these tools, attempts to ensure correct sizing of physical servers.
• The Application Service Management (ASM) group. In its first two years, the RightSizing initiative did not convince enough business groups to adopt virtualization, and the number of servers deployed in the data centers continued to grow. The ASM group formed in the summer of 2007 with the goal to gain operational efficiencies across a spectrum of business processes. One of ASM's subgroups is called ASM Capacity Management, which now is known as Services Management Division (SMD) Capacity Planning. This group's goal is to enforce virtualization as a solution to critical data center space and power-consumption issues.

Compute Utility Strategy and RightSizing Initiatives

The Compute Utility strategy and RightSizing initiative were the first attempt at encouraging business groups at Microsoft to adopt virtualization. The Compute Utility strategy aimed to change the way business groups think about computing services. The goal of the RightSizing initiative is to identify the servers that are good virtualization candidates and encourage business groups to move forward with replacing those physical servers with virtual machines.

Compute Utility Strategy

The goal of the Compute Utility strategy is to abstract the services that the Microsoft data centers' hardware provides. Rather than have a business unit address its computing requirements by purchasing a server, this strategy asks a business group to provide its computing-capacity requirements. Microsoft IT then determines whether a virtual or physical server can meet those requirements and provides the service. The Compute Utility strategy sought to create this level of abstraction for business groups to encourage them to purchase computing power and storage without worrying about the server hardware.

RightSizing Initiative

Microsoft IT developed the RightSizing initiative to ensure correct utilization of servers in the data center and in managed labs. Because significant underutilization occurs, one of the initiative's first tasks was for Microsoft IT to identify underutilized servers that might be good candidates for virtualization. To do this, Microsoft IT:

• Identified performance characteristics of the servers in the data centers. Microsoft IT used Microsoft Operations Manager 2005, and later, Microsoft System Center Operations Manager 2007, to collect the information. Over several months, the team collected CPU performance data, including the maximum level of CPU utilization, average CPU utilization, and minimum CPU utilization. Additionally, the team collected aggregate data from Microsoft Operations Manager to obtain monthly CPU utilization averages, and it categorized each server based on four "temperature" categories. Each temperature category has a defined value for the Mean CPU percentage (Avg Mean Value) and Max CPU percentage (Avg Max Value) CPU utilization levels. The Avg Max Value is a one-month average of the daily Maximum Values reported (sum of Maximum Value data points for the month / quantity of data points = Avg Max Value). The Avg Mean Value is a one-month average of the daily Average Values reported (sum of Average Value data points for the month / quantity of data points = Avg Mean Value). Table 1 shows the temperatures and values.

  Table 1. Identifying Server Utilization by Temperature

  Temperature	Mean CPU percentage	Max CPU percentage
  Permafrost	<=1	<=5
  Cold	<=5	<=20
  Warm	<=20	<=50
  Hot	>20	>50

  
  
  

  By using this performance data, the RightSizing team could identify Permafrost or Cold servers. These categories became part of every communication with the business groups to provide a consistent means of identifying potential virtualization candidates.

  Note: For details on how the RightSizing team identified the servers that were good virtualization candidates, see the Microsoft IT Showcase technical case study "Identifying Server Candidates for Virtualization."

• Developed application-specific recommendations for server virtualization. Over time, the RightSizing team identified which server applications were good virtualization candidates. During the early stages of the RightSizing initiative, all virtual machines ran Windows Server 2003 and Virtual Server 2005. Due to Virtual Server 2005 limitations, the initial recommendations focused on services, such as Web servers running Internet Information Services (IIS) and Web front-end servers for Windows® SharePoint® Services or Microsoft Office SharePoint Server. The RightSizing team recommended that these services move to virtual machines. As Microsoft IT implements Hyper-V™ technology on Windows Server® 2008, these recommendations are changing to include most server applications as virtualization candidates, including those that require a 64-bit operating system.

Communication of RightSizing to Business Groups

To drive adoption of virtualization, the RightSizing team developed a RightSizing scorecard that presents a comprehensive view of the current virtualization status and future virtualization possibilities. This scorecard provides two views:

• Summary view. The summary view provides high-level information related to several virtualization criteria. For example, the summary page includes graphs that show the server growth trend, the underused servers, CPU utilization, and virtual machine ratio. In each case, the graph shows the valuation changes over the past year and projects the impact of those changes for the next year.
• Business view. The business view reports on how each business unit is performing in comparison to other business groups. The categories for each business unit include the number of servers that it has deployed in the data centers (both the year-to-date amount and the current-month information) and the virtual machine ratio.

For each business unit, the scorecard provides a summary view that lists the number of servers it has deployed in both data centers and the IT-managed labs. Each view highlights the number and percentage of all virtual machines.

The Virtual Server Utility (VSU) Opportunities section is a key component in the business view. This section provides details, for three different options, on the cost savings available if the business unit replaces physical servers with virtual machines.

The three options are:

• Option A: Replacement of all end-of-life and end-of-warranty servers that have a temperature of Permafrost and Cold. End-of-life servers are servers that Microsoft IT no longer supports in its production environments for performance, supportability, or cost reasons.
• Option B: Option A, and replacement of all end-of-life and end-of-warranty servers for which no CPU utilization value exists.
• Option C: Option B, and replacement of 50 percent of retired servers that have a temperature of Permafrost and Cold.

Part of each business unit's scorecard lists the number of servers in each category and the potential savings that the business unit can achieve by replacing old servers with virtual machines rather than physical servers.

The RightSizing scorecard is available on a Microsoft intranet Web site, and upper management frequently reviews the reports at its meetings. The RightSizing scorecard drives virtualization adoption by providing the following:

• Data on the status of virtualization and data on the servers that are good virtualization candidates.
• Comparisons between business groups to promote competition to drive virtualization.
• Reports on missed virtualization opportunities. As part of the summary view, the scorecard highlights opportunities where the RightSizing team identified a server candidate for virtualization, but where the business group instead purchased a physical server.

Response to the RightSizing Initiative

The RightSizing initiative greatly increased virtualization visibility and deployment at Microsoft. By the end of June 2007, Microsoft IT had moved approximately 10 percent of servers to a virtual machine. One estimate is that the RightSizing and virtualization initiatives saved Microsoft $9.75 million US over two years.

However, the response to the RightSizing initiative was not completely positive. Some business groups aggressively adopted virtualization, whereas other groups did not. As a result, the number of physical servers deployed in the data centers continued to increase at a rate of approximately 2,000 servers per year. The RightSizing initiative continues to publish the scorecards on a monthly basis. However, by itself, the initiative did not resolve the data-center space issues.

Development of a Virtual Machine Costing Model

At the same time that the RightSizing initiative was encouraging business groups to consider virtualization, the IT Compute Utility team was developing its VSU service. This group defines the standards and implements all virtual machines running in the Microsoft data centers or IT-managed labs.

Microsoft IT finances its services by charging business groups with the cost of providing services. As part of the virtualization initiative, the IT Compute Utility group created a comparison for hosting a physical server versus a virtual machine. Table 2 shows the comparison.

Table 2. Comparing Physical Server and Virtual Machine costs

An additional benefit of a virtual machine is that the business unit does not pay for the cost of the underlying physical machine. The monthly hosting fee includes that cost. As a result, the business unit does not need to worry about replacing the physical hardware when purchasing a virtual machine, which increases savings significantly over the server hardware's life.

Application Service Management Group

The second step in implementing virtualization at Microsoft was with the ASM group's creation in the summer of 2007. The impetus for starting the ASM group was quite different from the impetus for creating the RightSizing team. By March 2007, the Microsoft data centers in Redmond essentially had reached their capacity for space and power, and adding more data-center space was cost prohibitive. At the same time, the experience gained from the RightSizing initiative showed the potential that virtualization had for reducing server deployment in data centers.

As a result, upper IT management created the ASM group and the Services Management Division (SMD) Capacity Management subgroup with a mandate to enforce virtualization. The SMD Capacity Management group is a capacity-planning and server-purchasing group that can enforce purchasing standards across multiple business groups. For the 23 largest of the current 27 groups that can place hardware orders, all orders now must pass through the SMD Capacity Management group, which evaluates all requests for replacing current servers and purchasing new ones. Because of the group's mandate to enforce virtualization, it assumes that all server deployments use a virtual machine unless a compelling reason exists to deploy physical hardware.

To encourage business groups to adopt virtualization, the SMD Capacity Management group:

• Provides incentives, such as free virtual machines. For fiscal year 2008 (FY08), the team provides a free virtual machine and free hosting for any business unit that replaces an existing server with a virtual machine. The only stipulation is that the business group must decommission one existing physical server in the data center. Midway through FY08, this program had provided more than 1,000 free virtual machines.
• Blocks all hardware purchases until it investigates virtualization for each server. The SMD Capacity Management team has made virtualization the default deployment option for all new and existing server replacements, placing the onus on the business group to prove that it needs a physical server.

The goal of the ASM group is to move 25 percent of the servers in the data centers and managed lab environments to virtual machines by June 2008, and 50 percent by June 2009. As of May 2008, Microsoft IT had deployed more than 3,000 virtual machines. As Microsoft IT deploys Hyper-V on Windows Server 2008, the expectation is that 90 percent of new server orders will deploy as virtual machines on Hyper-V.

Lessons Learned

Virtualization has the potential to provide significant cost savings for Microsoft IT. By reducing the physical servers in data centers, Microsoft can save millions of dollars in hardware purchases and power consumption. Additionally, by freeing space in the current data centers, Microsoft may avoid the cost of building a new data center.

Despite the benefits of virtualization, some issues have surfaced regarding full-scale adoption of this technology. The lessons learned include:

• Start implementing virtualization now. Although Microsoft IT started the RightSizing initiative well before its data centers were full, the continued preference of business groups for purchasing physical servers meant that data centers reached capacity before the ASM group enforced virtualization. However, data centers are at capacity, which makes it difficult even to deploy a physical server to host the virtual machines.
• Be creative and persistent in promoting the benefits of virtualization. At Microsoft, the RightSizing team developed a comprehensive scorecard with both summary views for upper management and very detailed data for IT administrators. The information includes comparisons between business groups and virtualization success stories. The IT Compute Utility team has developed an intranet site that provides current statistics on the virtualization efforts. The site also includes links that provide information about virtualization, reasons for choosing virtualization, and an application form from which a business unit can order a virtual machine.
• Implement a consistent approach for server capacity planning and virtualization across business groups. If each business group can purchase servers independently, and one or more business groups do not adopt virtualization, the rest of the business groups are likely to feel less pressured to use virtualization. By implementing a horizontal group such as ASM, Microsoft could enforce virtualization across all business groups.

Conclusion

The Microsoft data centers in Redmond are reaching capacity, while many servers in those data centers are operating at much less than full capacity. This is an increasingly common situation for many businesses.

Microsoft has identified server virtualization as one of the most important tools for addressing both concerns. By deploying multiple virtual machines on a single physical server, Microsoft IT can reduce data-center requirements for space and power significantly.

However, virtualization is a new technology. Therefore, business groups that are familiar with running their applications on physical servers may require incentives and encouragement to move their critical processes to virtual machines. At Microsoft, the Compute Utility strategy and RightSizing initiative started moving business groups toward virtualization. The ASM group is continuing this process by enforcing virtualization.

For More Information

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase

© 2008 Microsoft Corporation. All rights reserved.

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Hyper-V, SharePoint, Windows, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.