White Paper: Comparing the Power Utilization of Native and Virtual Exchange Environments

 

Dernière rubrique modifiée : 2010-05-13

Rob Simpson, Exchange Program Manager

June 2009

Résumé

Reducing or controlling the high cost of the power to run and cool computer hardware is a top priority for many organizations throughout the world. Many organizations are considering server virtualization solutions to reduce their server footprint and the associated power and cooling costs. Because the virtualization of Microsoft Exchange servers rarely results in a reduction of physical processors, there is some question whether there is significant hardware, power, cooling, or space savings from virtualizing correctly sized Exchange Server 2007 server roles. This study compared the power utilization of native and virtual Exchange server environments in a scenario in which the number of physical servers was reduced from eight to two, but the total number of logical processors and the amount of memory remained the same.

When you consider virtualization of an Exchange environment, power savings is only one of many factors to consider. Depending on your requirements, virtualization may not be a good fit for your Exchange environment. For more information, see the Exchange Team Blog article Should You Virtualize Your Exchange 2007 SP1 Environment?

Applies To

Microsoft Exchange Server 2007

Table of Contents

• Introduction

• Objectives

• Test Environment

  • Overview

  • Native Exchange Server Environment

  • Virtual Server Configuration

  • Storage Configuration

  • Load Server Configuration

  • Additional Power Utilization Configuration Parameters

  • Collection of Power Utilization Data

  • Test Facility

• Test Methodology

  • Exchange Mailbox Profiles

  • Exchange User Profiles

• Exchange Workload Simulation

• Power Utilization Results

  • Native Exchange Server Environment

  • Virtual Exchange Server Environment

  • Direct Attached Storage

  • Comparison of Native and Virtual Environments

• Conclusion

• Informations supplémentaires

  • Appendix A - Performance Results

Introduction

Reducing or controlling high power and cooling costs has become a top priority for many organizations throughout the world. These organizations are looking at implementing server virtualization solutions to reduce their server footprint and the associated power and cooling costs. One of the most common virtualization scenarios is to take many underused physical servers (servers that use less than 20 percent of processor, memory, and other server resources) and merge them on virtual operating systems that are hosted on fewer and larger physical servers. A reduction in the number of servers, the number of processors, the amount of local disk space, and so on, often significantly reduces power consumption. This scenario is very common with Web or application server farms that deploy low-use applications on a dedicated physical server for security, administration, or cost center reasons.

Exchange servers running in a correctly sized enterprise environment typically do not fit into the category of “underused," and virtualization of enterprise Exchange servers rarely reduces the number of physical processors. Therefore, there is some question whether power, cooling, or space is significantly reduced by virtualization of these environments.

This study provides the data that is needed to determine whether virtualization of Exchange server roles provides any power savings.

Objectives

Several hardware configurations were tested to provide data points for power utilization in physical and virtual deployments. The test data was used to provide a comparative analysis of the physical and virtualized hardware deployments. The following list describes the objectives of the study:

• Determine the power consumption over a 24-hour period for eight physical servers together with the following Exchange server roles:

  • Two Exchange 2007 servers with the Client Access role installed.

  • Two Exchange 2007 servers with the Hub Transport role installed.

  • Four Exchange 2007 servers with the Mailbox role installed.

• Determine the power consumption over a 24-hour period for two physical Hyper-V root servers together with the following virtual Exchange server roles:

  • Two Exchange 2007 servers with the Client Access server role installed

  • Two Exchange 2007 servers with the Hub Transport server role installed

  • Four Exchange 2007 servers with the Mailbox server role installed

• Determine the power consumption of two common Direct Attached Storage (DAS) solutions in a 24-hour period.

• Compare the server power utilization of the tested physical and virtual Exchange server environments.

• Compare the total power utilization (server and storage) of physical and virtual Exchange server environments.

Retour au début

Test Environment

Overview

The test environment for this study consisted of a native (physical) Exchange server environment and a virtual Exchange server environment. Each environment supported 16,000 active mailboxes that were running a LoadGen heavy user profile. Each environment contained a mix of MAPI, Outlook Web App, and Exchange ActiveSync Client Access types. (For a full description of the workload, see the "Test Methodology" section.)

Visual representation of hardware that was used in the test environment

Native Exchange Server Environment

The native Exchange server environment consisted of two Client Access servers, two Hub Transport servers, four Mailbox servers, and two domain controllers all running a 64-bit version of Windows Server 2008. All physical Exchange servers were deployed on Hewlett-Packard (HP) DL380 G5 servers that had appropriate memory and the disk configurations shown in Table 1. The Mailbox servers were connected to HP Direct Attached Storage (DAS) that had both 300-GB SAS (MSA60s) and 146-GB SAS (MSA70s) disk types represented. (See the "Storage Configuration" section for additional details.)

Table 1 Summary of hardware that was used in the native environment

Servers	Disk	Server role / names
HP ProLiant DL380 G5 2.8GHz Dual Core (4 processor cores) 8 GB RAM	2 x 146-GB SAS local drives	Client Access / CAS1 and CAS2
HP ProLiant DL380 G5 2.8GHz Dual Core (4 processor cores) 4 GB RAM	6 x 146-GB SAS local drives	Hub Transport / HUB1 and HUB2
HP ProLiant DL380 G5 2.8GHz Dual Core (4 processor cores) 16 GB RAM P800 SAS Controller	2 x 146-GB SAS local drives and 2 x MSA70 with 25x146-GB SAS or 2 x MSA60 with 12x300-GB SAS	Mailbox / MBX1, MBX2, MBX3, MBX4

Retour au début

Virtual Server Configuration

The virtual server environment consisted of two physical root servers that were running a 64-bit version of Windows Server 2008 and the release to manufacturing (RTM) version of Windows Server 2008 Hyper-V. (There were no Hyper-V hotfixes at the time of this test.) For more information about the RTM version of Windows Server 2008 Hyper-V, see Microsoft Knowledge Base article 950050, Description of the update for the release version of the Hyper-V technology for Windows Server 2008.

All physical root servers were deployed on HP DL580 G5 servers with appropriate memory and the disk configurations shown in Table 2. Microsoft System Center Virtual Machine Manager was used to convert each native Exchange server into a guest virtual machine. One virtual Client Access server, one virtual Hub Transport server, and two virtual Mailbox servers were deployed on each physical Hyper-V root server. The original DAS was then transferred to the root servers and configured as SCSI pass-through storage. All storage and Exchange server configurations were identical in the native and virtual environments. The two domain controllers were not in the scope of this test, and they remained physical servers.

Table 2 Summary of hardware that was used in the virtual environment

Servers	Disk	Virtual Machine	Server role / names
HP ProLiant DL580 G5 2.8GHz Quad Core (16 CPU) 48 GB RAM P800 SAS Controller	10 x 146-GB SAS local drives 4 x MSA60 12x300-GB SAS	Four virtual processors, 8 GB Four virtual processors, 4 GB Four virtual processors, 16 GB Four virtual processors, 16 GB	Client Access / CAS1 Hub Transport / HUB1 Mailbox / MBX1 and MBX2
HP ProLiant DL580 G5 2.8GHz Quad Core (16 CPU) 48 GB RAM P800 SAS Controller	10 x 146-GB SAS local drives 4 x MSA70 25x146-GB SAS	Four virtual processors, 8 GB Four virtual processors, 4 GB Four virtual processors, 16 GB Four virtual processors, 16 GB	Client Access / CAS2 Hub Transport / HUB2 Mailbox / MBX3 and MBX4

Retour au début

Storage Configuration

Two common HP DAS solutions were tested as part of this study. The first solution used two HP MSA70 storage enclosures with 48 146-GB (2.5") hard disk drives per Mailbox server. The second solution used two HP MSA60 storage enclosures with 24 300-GB SAS (3.5") hard disk drives per Mailbox server. Both solutions were tested with 4,000 mailboxes per server and an initial mailbox size of 200 MB. For more information about the storage solutions that were used in this study, see Table 3.

Table 3 Summary of storage solutions

Enclosure type	Number of spindles	LUNs	SGs/DBs	Mailboxes
HP MSA70	48 x 146-GB SAS (12 per LUN)	3 x 800-GB RAID10 DB 1 x 800GB RAID10 LOG	30 / 30	4,000 x 200 MB
HP MSA60	24 x 300-GB SAS (6 per LUN)	3 x 800-GB RAID10 DB 1 x 800-GB RAID10 LOG	30 / 30	4,000 x 200 MB

Load Server Configuration

The server configuration for load testing consisted of sixteen servers (load servers) running a 64-bit version of Windows Server 2008 and Microsoft Exchange Load Generator (v8.02.0045). All load servers were deployed on HP DL380 G4 servers with the memory and the disk configurations shown in Table 4.

Table 4 Summary of hardware used for load generation

Servers	Disk	Server role
HP ProLiant DL380 G4 2.8GHz Dual Core (four processor cores) 8 GB RAM	2 x 146-GB SAS local drives	LOAD1-LOAD16

Retour au début

Additional Power Utilization Configuration Parameters

The latest BIOS versions for the HP ProLiant DL380 G5 and DL580 G5 servers that were used in this study support advanced power regulation settings as defined below. All servers were set to OS Control Mode. (The default configuration for these servers is HP Dynamic Power Savings Mode.)

BIOS advanced power regulation settings included:

• HP Dynamic Power Savings Mode: Automatically varies processor speed and power usage based on processor utilization. Allows for reducing overall power consumption with little or no effect on performance. Does not require operating system support.

• HP Static Low Power Mode: Reduces processor speed and power usage. Guarantees a reduced maximum power usage for the system. Performance effects will be bigger for environments that have more processor utilization.

• HP Static High Performance Mode: Processors will always run in their maximum power/performance state regardless of the operating system power management policy.

• OS Control Mode: Processors always run in their maximum power/ performance state unless the operating system is using a power management policy.

In addition to the BIOS settings, all native Exchange servers and Hyper-V root servers had the following Windows Server 2008 power configuration settings applied. These settings change P-state parameters so that they drop to a lower frequency state more aggressively during light loads. We expected this to provide better power utilization during the evening period of our 24-hour test.

powercfg /setpossiblevalue /sub_processor /procperf 2 binary 640364000000a0860100a08601001e00000032000000

powercfg /setactive scheme_balanced

For more information about P-State parameters, see the Windows Server Performance Team Blog article, Configuring Windows Server 2008 Power Parameters for Increased Power Efficiency.

Retour au début

Collection of Power Utilization Data

Power consumption for all Exchange servers and DAS servers was monitored by using Smart-Watt power monitoring devices. These devices were in-line between the server power supply and the rack power distribution unit and collected average watt consumption over five-minute intervals. The monitoring devices read the power consumption millions of times per minute and produced an average for the five-minute period. Each power read included a time stamp, the number of watts being consumed, and the total watt hours consumed to date. The data from all devices was collected and stored in a Microsoft SQL Server database for easy analysis. For more information about Smart-Watt devices, see the Smart Works Web site.

Test Facility

This testing was conducted at the Microsoft Enterprise Engineering Center (EEC), a state-of-the-art data center on the Microsoft main campus in Redmond, Washington. With over $100M in hardware and networking equipment from top manufacturers, the EEC can replicate almost any production environment. Many of our most important customers visit the EEC to see how prereleases of Microsoft products will perform in a production environment before the customers deploy these products in their own environments. By visiting the EEC, customers can influence the final release, reduce uncertainty, and reduce their deployment costs.

Customer validation testing at the EEC also lets Microsoft product groups listen directly to customers, learn about real-world applications of our technologies, and work with customers to find defects in the software before it’s released to the market.

Test Methodology

Exchange Mailbox Profiles

Each Mailbox server was configured with 4,000 mailboxes for a total of 16,000 mailboxes in the environment. The mailboxes were initialized with an average size of 200 MB using Load Generator (LoadGen).

Exchange User Profiles

Because the main goal of this study was to examine power utilization from a typical enterprise Exchange environment, we selected an Exchange user protocol mix that would represent an average enterprise environment and sufficiently load all Exchange server roles (see Table 5).

The client protocol mix was 75 percent MAPI (Outlook 2007) and 25 percent Outlook Web Access. Exchange ActiveSync was run against 25 percent of the mailboxes. The incoming e-mail message rate was set to seven messages per second.

Table 5 Summary of Exchange user profiles tested

Protocol	LoadGen profile	Number of users
MAPI	Outlook 2007 (Online) LoadGen Heavy Profile	3,000 per Mailbox server (75%)
Outlook Web Access	Outlook Web Access Enterprise 2007 LoadGen Default Script	1,000 per Mailbox server (25%)
Exchange ActiveSync	Exchange ActiveSync v12 DirectPush LoadGen Default Script	1,000 per Mailbox server (25%)
SMTP	Default SMTP Script (x3) Default SMTP Messages	Mail delivered to 16,000 users at ~seven messages/sec

Retour au début

Exchange Workload Simulation

To obtain accurate power utilization data over a 24-hour period, an Exchange workload was applied during testing to mimic that of a typical enterprise environment. The 24-hour workday was divided into four periods as shown in Table 6. The early morning workload consisted primarily of Exchange ActiveSync and Outlook Web Access activity. This period was followed by a peak logon of Outlook 2007 MAPI clients and heavy MAPI loads throughout the day. During the evening period, the MAPI load decreased, leaving primarily Exchange ActiveSync and Outlook Web Access activity. SMTP mail was configured for incoming delivery during the first 16 hours. During the evening period, online maintenance was scheduled to run for 6 of the 8 hours. Although this workload does not match a standard enterprise production workload, it is a good balance between using a representative workload and minimizing the complexity of running the test.

Table 6 Summary of Exchange workload over a 24-hour period

Period	Duration	Workload
Early Morning (6 A.M. - 8 A.M.)	2 hrs	Exchange ActiveSync and Outlook Web Access Incoming SMTP
Workday (8 A.M. - 6 P.M.)	10 hrs	Outlook 2007 (MAPI), Exchange ActiveSync, and Outlook Web Access Incoming SMTP
Evening (6 P.M. - 10 P.M.)	4 hrs	Exchange ActiveSync and Outlook Web Access Incoming SMTP
Overnight (10 P.M. - 6 A.M.)	8 hrs	Exchange Information Store (IS) maintenance
Total	24 hrs

Retour au début

Visual representation of processor utilization over a 24-hour period

Power Utilization Results

Native Exchange Server Environment

When measured over a 24-hour period, the native Exchange servers consumed between 210 and 274 average watts. This data is summarized in Table 7. During the 24-hour period, the environment with eight physical servers consumed a total of 1,962 average watts. The projected annual power consumption for the native environment is 17,184 kilowatt hours (kWh)/year.

Table 7 Power utilization for the native Exchange server environment

Role	Average watts (24 hrs)	Number of servers	Total average watts (24 hrs)	kWh/year (projected)
Client Access	210	2	420	3,679
Hub Transport	223	2	446	3,907
Mailbox	274	4	1,096	9,601
Total	Not applicable	8	1,962	17,187

Retour au début

Virtual Exchange Server Environment

A physical Hyper-V root server configured with four guest virtual machines running Exchange 2007 roles (including one Client Access server, one Hub Transport server, and two Mailbox servers) consumed 491 average watts, measured over a 24-hour period. This data is summarized in Table 8. The total average number of watts consumed by the virtual environment, measured over a 24-hour period, was 982. The projected annual power consumption for the virtual environment was 8,602 kWh/year.

Table 8 Power utilization for the Hyper-V host server

Role	Average watts (24 hrs)	Number of Servers	Total average watts (24 hrs)	kWh/year (projected)
Hyper-V	491	2	982	8,602

Direct Attached Storage

The HP MSA60 storage solution, which consisted of 96 spindles in eight enclosures, consumed 672 average watts. This data is summarized in Table 9. The projected annual power consumption for this storage solution was 5,888 kWh/year.

Table 9 Power utilization for the native Exchange server environment using MSA60s

Role	Average watts (24 hrs)	Number of enclosures	Total average watts (24 hrs)	kWh/year (projected)
MSA60	84	8	672	5,887

The HP MSA70 storage solution, which consisted of 192 spindles in eight enclosures, consumed 952 average watts. This data is summarized in Table 10. The projected annual power consumption for this storage solution was 8,340 kWh/year.

Table 10 Power utilization for the native Exchange server environment using MSA70s

Role	Average watts (24 hrs)	Number of Enclosures	Total average watts (24 hrs)	kWh/year (projected)
MSA70	119	8	952	8340

Retour au début

Comparison of Native and Virtual Environments

After these tests were finished, the data was analyzed and compared. The native Exchange 2007 servers, without accounting for storage utilization, consumed 1,962 average watts during the 24-hour test period. After the eight native Exchange 2007 servers were converted into eight virtual Exchange 2007 servers running on two physical Hyper-V root servers, the two root servers consumed 982 average watts during the 24-hour test period. This virtualization scenario resulted in a 50 percent reduction in power consumption as shown in the following figure. This translates into a projected savings of 8,582 kWh/year.

The MSA60 storage solution that was tested consumed 672 average watts during the 24-hour test period. If the storage power consumption is added to the server power consumption, 2,634 average watts were consumed by the native server environment, and 1,654 average watts were consumed by the virtual server environment. When the MSA60 storage solution was included, the virtualization scenario had a 37 percent reduction in power consumption.

The MSA70 storage solution consumed 948 average watts during the 24-hour test period. If storage power consumption is added to server power consumption, 2,910 average watts were consumed by the native server environment and 1,930 average watts were consumed by the virtual server environment. When the MSA70 storage solution was included, the virtualization scenario had a 34 percent reduction in power consumption.

Comparison of average watts consumed by native and virtual environments

Retour au début

Conclusion

This study examined power utilization of native and virtual Exchange 2007 environments in a scenario where physical servers were reduced from 8 to 2 but the total number of logical processors remained constant at 32. There was no processor core consolidation, and storage power utilization was not included. In this scenario, there was a 50 percent reduction in server power utilization and a projected savings of 8,582 kWh/year.

When you examine the power savings of the total solution, including servers and storage, you see a 37 percent reduction in power utilization when the MSA60 storage solution is deployed and a 34 percent reduction in power utilization when the MSA70 storage solution is deployed.

Notice that power utilization for the largest DAS solution (MSA70) accounts for less than 50 percent of the total power consumed by the virtual environment and less than 35 percent of the total power consumed by the native environment. When DAS solutions are used, storage is not the primary consumer of power.

Important :
This study does not examine the effect of virtualization on reducing cooling requirements. In many environments, cooling can cause significant power consumption. Therefore, it should be included in any cost analysis. The Microsoft HyperGreen tool can be used to help estimate reduction in cooling costs for virtualization scenarios.

Although this study showed that virtualization of Exchange servers can generate power savings, power savings is not the only factor to consider when you evaluate whether virtualization of Exchange is appropriate for a specific environment. Adding virtualization to an Exchange environment can introduce additional complexity in several areas including backup, monitoring, and storage configurations. Virtualization of any Exchange solution should be carefully considered and planned, and virtualization scenarios should be tested with server loads that mimic the loads in the actual production environment. We recommend that you review the references in the "Additional Information" section before you consider implementation of Exchange on virtualized hardware.

Retour au début

Informations supplémentaires

Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments

Virtualisation d'Exchange 2007 SP1

The Microsoft HyperGreen tool

Windows Server Virtualization Validation Program

Windows Server Hyper-V

Hyper-V Planning and Deployment Guide

Microsoft Assessment and Planning Toolkit

Hyper-V Getting Started Guide

Performance Tuning Guidelines for Windows Server 2008 (Virtualization Section)

Hyper-V Performance FAQ

Monitoring Hyper-V Performance

Microsoft System Center Virtual Machine Manager

Appendix A - Performance Results

Note   The following performance results are provided for information and should not be used for sizing guidance or to make performance recommendations for virtualizing Exchange environments. For more information, see Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments.

Table A1 Comparison of key performance metrics for native and virtual Client Access servers

Counter	Native	Virtual
MSExchange OWA\Current Unique Users	2,000	2,000
MSExchange OWA\Average Response Time	29.5	29.0
MSExchange OWA\Requests/sec	55.7	53.1
MSExchange ActiveSync\Average Request Time	93	111
MSExchange ActiveSync\Requests/sec	14.6	15.2
% Processor Time (4 processor cores)	36%	40%*
Hyper-V Hypervisor Logical Processor\ %Guest Run Time	Not applicable	9%

*The processor was measured in the guest operating system. This value is inaccurate when it is measured in a Hyper-V guest operating system. See Table A4 for Hyper-V host data.

Retour au début

Table A2 Comparison of key performance metrics for native and virtual Hub Transport servers

Counter	Native	Virtual
Logical Disk(*)\Average Disk sec/read	0.002	0.008*
Logical Disk(*)\Average Disk sec/write	0.001	0.013*
MSExchange Transport SMTP Receive\Messages received/sec	5.5	5.2
% Processor Time (4 processor cores)	29%	32%**
Hyper-V Hypervisor Logical Processor\ %Guest Run Time	Not applicable	7%

*These counter results may be higher than in the native environment because they were in a .vhd file. This file was located on a disk that was shared with other guest virtual machine vhds (virtual hard drives).

**The processor was measured in the guest operating system. This value was shown to be inaccurate when measured in a Hyper-V guest operating system. See Table A4 for Hyper-V host data.

Retour au début

Table A3 Comparison of key performance metrics for native and virtual Mailbox servers

Counter	Native	Virtual
Disk Transfers/sec	1,717	1,752
IOPS/user	0.43	0.44
Logical Disk(*)\Average Disk sec/read	0.007	0.008
Logical Disk(*)\Average Disk sec/write	0.001	0.001
Disk Read Bytes/sec	11,827,000	12,666,000
Disk Write Bytes/sec	15,325,000	15,333,000
MSExchangeIS\RPC Average Latency	2.5	5.8*
MSExchangeIS\RPC Requests	4.6	8.3
MSExchangeIS Client\RPC Operations/sec (total)	3,175	2,934
MSExchangeIS Client\RPC Operations/sec (EAS)	685	545
MSExchangeIS Client\RPC Operations/sec (OWA)	474	357
MSExchangeIS Client\RPC Operations/sec (MAPI)	845	846
MSExchangeIS Client\RPC Operations/sec (CI)	343	314
MSExchangeIS Client\RPC Operations/sec (Transport)	714	695
% Processor Time (4 processor cores)	48%	47%*
Hyper-V Hypervisor Logical Processor\ %Guest Run Time	Not applicable	11%

*RPC Average Latency is inaccurate when it is measured in a Hyper-V guest operating system.

**The processor was measured in the guest operating system. This value has been shown to be inaccurate when measured in a Hyper-V guest operating system (see Table A4).

Table A4 Key performance metrics for the Hyper-V Root Servers

Counter	Hyper-V root
Hyper-V Logical Processor Count	16
Hyper-V Hypervisor Logical Processor\ %Guest Run Time	37%
Hyper-V Hypervisor Logical Processor\ %Hypervisor Run Time	8%
Hyper-V Hypervisor Logical Processor\ %Idle Run Time	55%
Hyper-V Hypervisor Logical Processor\ %Total Run Time	45%
Hyper-V Hypervisor Logical Processor \Context Switches / sec	7,102,141

Retour au début