Export (0) Print
Expand All
Expand Minimize
4 out of 5 rated this helpful - Rate this topic

White Paper: Comparing the Power Utilization of Native and Virtual Exchange Environments

 

Topic Last Modified: 2010-05-13

Rob Simpson, Exchange Program Manager

June 2009

Reducing or controlling the high cost of the power to run and cool computer hardware is a top priority for many organizations throughout the world. Many organizations are considering server virtualization solutions to reduce their server footprint and the associated power and cooling costs. Because the virtualization of Microsoft Exchange servers rarely results in a reduction of physical processors, there is some question whether there is significant hardware, power, cooling, or space savings from virtualizing correctly sized Exchange Server 2007 server roles. This study compared the power utilization of native and virtual Exchange server environments in a scenario in which the number of physical servers was reduced from eight to two, but the total number of logical processors and the amount of memory remained the same.

When you consider virtualization of an Exchange environment, power savings is only one of many factors to consider. Depending on your requirements, virtualization may not be a good fit for your Exchange environment. For more information, see the Exchange Team Blog article Should You Virtualize Your Exchange 2007 SP1 Environment?

Microsoft Exchange Server 2007

Reducing or controlling high power and cooling costs has become a top priority for many organizations throughout the world. These organizations are looking at implementing server virtualization solutions to reduce their server footprint and the associated power and cooling costs. One of the most common virtualization scenarios is to take many underused physical servers (servers that use less than 20 percent of processor, memory, and other server resources) and merge them on virtual operating systems that are hosted on fewer and larger physical servers. A reduction in the number of servers, the number of processors, the amount of local disk space, and so on, often significantly reduces power consumption. This scenario is very common with Web or application server farms that deploy low-use applications on a dedicated physical server for security, administration, or cost center reasons.

Exchange servers running in a correctly sized enterprise environment typically do not fit into the category of “underused," and virtualization of enterprise Exchange servers rarely reduces the number of physical processors. Therefore, there is some question whether power, cooling, or space is significantly reduced by virtualization of these environments.

This study provides the data that is needed to determine whether virtualization of Exchange server roles provides any power savings.

Several hardware configurations were tested to provide data points for power utilization in physical and virtual deployments. The test data was used to provide a comparative analysis of the physical and virtualized hardware deployments. The following list describes the objectives of the study:

  • Determine the power consumption over a 24-hour period for eight physical servers together with the following Exchange server roles:
    • Two Exchange 2007 servers with the Client Access role installed.
    • Two Exchange 2007 servers with the Hub Transport role installed.
    • Four Exchange 2007 servers with the Mailbox role installed.
  • Determine the power consumption over a 24-hour period for two physical Hyper-V root servers together with the following virtual Exchange server roles:
    • Two Exchange 2007 servers with the Client Access server role installed
    • Two Exchange 2007 servers with the Hub Transport server role installed
    • Four Exchange 2007 servers with the Mailbox server role installed
  • Determine the power consumption of two common Direct Attached Storage (DAS) solutions in a 24-hour period.
  • Compare the server power utilization of the tested physical and virtual Exchange server environments.
  • Compare the total power utilization (server and storage) of physical and virtual Exchange server environments.

Return to top

The test environment for this study consisted of a native (physical) Exchange server environment and a virtual Exchange server environment. Each environment supported 16,000 active mailboxes that were running a LoadGen heavy user profile. Each environment contained a mix of MAPI, Outlook Web App, and Exchange ActiveSync Client Access types. (For a full description of the workload, see the "Test Methodology" section.)

noteNote:
In this white paper, "native" refers to a Windows installation without Hyper-V enabled; "root" refers to the parent partition inside a Windows configuration with Hyper-V enabled; and "guest virtual machine" refers to the child partition that is hosted on the root (parent) partition of Windows.
Visual representation of hardware used in test

The native Exchange server environment consisted of two Client Access servers, two Hub Transport servers, four Mailbox servers, and two domain controllers all running a 64-bit version of Windows Server 2008. All physical Exchange servers were deployed on Hewlett-Packard (HP) DL380 G5 servers that had appropriate memory and the disk configurations shown in Table 1. The Mailbox servers were connected to HP Direct Attached Storage (DAS) that had both 300-GB SAS (MSA60s) and 146-GB SAS (MSA70s) disk types represented. (See the "Storage Configuration" section for additional details.)

Table 1 Summary of hardware that was used in the native environment

Servers Disk Server role / names

HP ProLiant DL380 G5

2.8GHz Dual Core (4 processor cores)

8 GB RAM

2 x 146-GB SAS local drives

Client Access / CAS1 and CAS2

HP ProLiant DL380 G5

2.8GHz Dual Core (4 processor cores)

4 GB RAM

6 x 146-GB SAS local drives

Hub Transport / HUB1 and HUB2

HP ProLiant DL380 G5

2.8GHz Dual Core (4 processor cores)

16 GB RAM

P800 SAS Controller

2 x 146-GB SAS local drives and 2 x MSA70 with 25x146-GB SAS

or

2 x MSA60 with 12x300-GB SAS

Mailbox / MBX1, MBX2, MBX3, MBX4

Return to top

The virtual server environment consisted of two physical root servers that were running a 64-bit version of Windows Server 2008 and the release to manufacturing (RTM) version of Windows Server 2008 Hyper-V. (There were no Hyper-V hotfixes at the time of this test.) For more information about the RTM version of Windows Server 2008 Hyper-V, see Microsoft Knowledge Base article 950050, Description of the update for the release version of the Hyper-V technology for Windows Server 2008.

All physical root servers were deployed on HP DL580 G5 servers with appropriate memory and the disk configurations shown in Table 2. Microsoft System Center Virtual Machine Manager was used to convert each native Exchange server into a guest virtual machine. One virtual Client Access server, one virtual Hub Transport server, and two virtual Mailbox servers were deployed on each physical Hyper-V root server. The original DAS was then transferred to the root servers and configured as SCSI pass-through storage. All storage and Exchange server configurations were identical in the native and virtual environments. The two domain controllers were not in the scope of this test, and they remained physical servers.

Table 2 Summary of hardware that was used in the virtual environment

Servers Disk Virtual Machine Server role / names

HP ProLiant DL580 G5

2.8GHz Quad Core (16 CPU)

48 GB RAM

P800 SAS Controller

10 x 146-GB SAS local drives

4 x MSA60 12x300-GB SAS

Four virtual processors, 8 GB

Four virtual processors, 4 GB

Four virtual processors, 16 GB

Four virtual processors, 16 GB

Client Access / CAS1

Hub Transport / HUB1

Mailbox / MBX1 and MBX2

HP ProLiant DL580 G5

2.8GHz Quad Core (16 CPU)

48 GB RAM

P800 SAS Controller

10 x 146-GB SAS local drives

4 x MSA70 25x146-GB SAS

Four virtual processors, 8 GB

Four virtual processors, 4 GB

Four virtual processors, 16 GB

Four virtual processors, 16 GB

Client Access / CAS2

Hub Transport / HUB2

Mailbox / MBX3 and MBX4

Return to top

Two common HP DAS solutions were tested as part of this study. The first solution used two HP MSA70 storage enclosures with 48 146-GB (2.5") hard disk drives per Mailbox server. The second solution used two HP MSA60 storage enclosures with 24 300-GB SAS (3.5") hard disk drives per Mailbox server. Both solutions were tested with 4,000 mailboxes per server and an initial mailbox size of 200 MB. For more information about the storage solutions that were used in this study, see Table 3.

Table 3 Summary of storage solutions

Enclosure type Number of spindles LUNs SGs/DBs Mailboxes

HP MSA70

48 x 146-GB SAS

(12 per LUN)

3 x 800-GB RAID10 DB

1 x 800GB RAID10 LOG

30 / 30

4,000 x 200 MB

HP MSA60

24 x 300-GB SAS

(6 per LUN)

3 x 800-GB RAID10 DB

1 x 800-GB RAID10 LOG

30 / 30

4,000 x 200 MB

The server configuration for load testing consisted of sixteen servers (load servers) running a 64-bit version of Windows Server 2008 and Microsoft Exchange Load Generator (v8.02.0045). All load servers were deployed on HP DL380 G4 servers with the memory and the disk configurations shown in Table 4.

Table 4 Summary of hardware used for load generation

Servers Disk Server role

HP ProLiant DL380 G4

2.8GHz Dual Core (four processor cores)

8 GB RAM

2 x 146-GB SAS local drives

LOAD1-LOAD16

Return to top

The latest BIOS versions for the HP ProLiant DL380 G5 and DL580 G5 servers that were used in this study support advanced power regulation settings as defined below. All servers were set to OS Control Mode. (The default configuration for these servers is HP Dynamic Power Savings Mode.)

BIOS advanced power regulation settings included:

  • HP Dynamic Power Savings Mode: Automatically varies processor speed and power usage based on processor utilization. Allows for reducing overall power consumption with little or no effect on performance. Does not require operating system support.
  • HP Static Low Power Mode: Reduces processor speed and power usage. Guarantees a reduced maximum power usage for the system. Performance effects will be bigger for environments that have more processor utilization.
  • HP Static High Performance Mode: Processors will always run in their maximum power/performance state regardless of the operating system power management policy.
  • OS Control Mode: Processors always run in their maximum power/ performance state unless the operating system is using a power management policy.

In addition to the BIOS settings, all native Exchange servers and Hyper-V root servers had the following Windows Server 2008 power configuration settings applied. These settings change P-state parameters so that they drop to a lower frequency state more aggressively during light loads. We expected this to provide better power utilization during the evening period of our 24-hour test.

powercfg /setpossiblevalue /sub_processor /procperf 2 binary 640364000000a0860100a08601001e00000032000000

powercfg /setactive scheme_balanced

For more information about P-State parameters, see the Windows Server Performance Team Blog article, Configuring Windows Server 2008 Power Parameters for Increased Power Efficiency.

Return to top

Power consumption for all Exchange servers and DAS servers was monitored by using Smart-Watt power monitoring devices. These devices were in-line between the server power supply and the rack power distribution unit and collected average watt consumption over five-minute intervals. The monitoring devices read the power consumption millions of times per minute and produced an average for the five-minute period. Each power read included a time stamp, the number of watts being consumed, and the total watt hours consumed to date. The data from all devices was collected and stored in a Microsoft SQL Server database for easy analysis. For more information about Smart-Watt devices, see the Smart Works Web site.

This testing was conducted at the Microsoft Enterprise Engineering Center (EEC), a state-of-the-art data center on the Microsoft main campus in Redmond, Washington. With over $100M in hardware and networking equipment from top manufacturers, the EEC can replicate almost any production environment. Many of our most important customers visit the EEC to see how prereleases of Microsoft products will perform in a production environment before the customers deploy these products in their own environments. By visiting the EEC, customers can influence the final release, reduce uncertainty, and reduce their deployment costs.

Customer validation testing at the EEC also lets Microsoft product groups listen directly to customers, learn about real-world applications of our technologies, and work with customers to find defects in the software before it’s released to the market.

Each Mailbox server was configured with 4,000 mailboxes for a total of 16,000 mailboxes in the environment. The mailboxes were initialized with an average size of 200 MB using Load Generator (LoadGen).

Because the main goal of this study was to examine power utilization from a typical enterprise Exchange environment, we selected an Exchange user protocol mix that would represent an average enterprise environment and sufficiently load all Exchange server roles (see Table 5).

The client protocol mix was 75 percent MAPI (Outlook 2007) and 25 percent Outlook Web Access. Exchange ActiveSync was run against 25 percent of the mailboxes. The incoming e-mail message rate was set to seven messages per second.

Table 5 Summary of Exchange user profiles tested

Protocol LoadGen profile Number of users

MAPI

Outlook 2007 (Online) LoadGen Heavy Profile

3,000 per Mailbox server (75%)

Outlook Web Access

Outlook Web Access Enterprise 2007 LoadGen Default Script

1,000 per Mailbox server (25%)

Exchange ActiveSync

Exchange ActiveSync v12 DirectPush LoadGen Default Script

1,000 per Mailbox server (25%)

SMTP

Default SMTP Script (x3) Default SMTP Messages

Mail delivered to 16,000 users at ~seven messages/sec

Return to top

To obtain accurate power utilization data over a 24-hour period, an Exchange workload was applied during testing to mimic that of a typical enterprise environment. The 24-hour workday was divided into four periods as shown in Table 6. The early morning workload consisted primarily of Exchange ActiveSync and Outlook Web Access activity. This period was followed by a peak logon of Outlook 2007 MAPI clients and heavy MAPI loads throughout the day. During the evening period, the MAPI load decreased, leaving primarily Exchange ActiveSync and Outlook Web Access activity. SMTP mail was configured for incoming delivery during the first 16 hours. During the evening period, online maintenance was scheduled to run for 6 of the 8 hours. Although this workload does not match a standard enterprise production workload, it is a good balance between using a representative workload and minimizing the complexity of running the test.

Table 6 Summary of Exchange workload over a 24-hour period

Period Duration Workload

Early Morning

(6 A.M. - 8 A.M.)

2 hrs

Exchange ActiveSync and Outlook Web Access

Incoming SMTP

Workday

(8 A.M. - 6 P.M.)

10 hrs

Outlook 2007 (MAPI), Exchange ActiveSync, and Outlook Web Access

Incoming SMTP

Evening

(6 P.M. - 10 P.M.)

4 hrs

Exchange ActiveSync and Outlook Web Access

Incoming SMTP

Overnight

(10 P.M. - 6 A.M.)

8 hrs

Exchange Information Store (IS) maintenance

Total

24 hrs

Return to top

Processor Ulilization over a 24 hour period

When measured over a 24-hour period, the native Exchange servers consumed between 210 and 274 average watts. This data is summarized in Table 7. During the 24-hour period, the environment with eight physical servers consumed a total of 1,962 average watts. The projected annual power consumption for the native environment is 17,184 kilowatt hours (kWh)/year.

noteNote:
To calculate kW/year, the following equation was used: KWh/year = ((Total Average Watts * 24) / 1000) * 365

Table 7 Power utilization for the native Exchange server environment

Role Average watts (24 hrs) Number of servers Total average watts (24 hrs) kWh/year (projected)

Client Access

210

2

420

3,679

Hub Transport

223

2

446

3,907

Mailbox

274

4

1,096

9,601

Total

Not applicable

8

1,962

17,187

Return to top

A physical Hyper-V root server configured with four guest virtual machines running Exchange 2007 roles (including one Client Access server, one Hub Transport server, and two Mailbox servers) consumed 491 average watts, measured over a 24-hour period. This data is summarized in Table 8. The total average number of watts consumed by the virtual environment, measured over a 24-hour period, was 982. The projected annual power consumption for the virtual environment was 8,602 kWh/year.

Table 8 Power utilization for the Hyper-V host server

Role Average watts (24 hrs) Number of Servers Total average watts (24 hrs) kWh/year (projected)

Hyper-V

491

2

982

8,602

The HP MSA60 storage solution, which consisted of 96 spindles in eight enclosures, consumed 672 average watts. This data is summarized in Table 9. The projected annual power consumption for this storage solution was 5,888 kWh/year.

noteNote:
The actual test environment consisted of four MSA60s and four MSA70s. For comparison, we calculated power consumption as if the complete solution used MSA60s.

Table 9 Power utilization for the native Exchange server environment using MSA60s

Role Average watts (24 hrs) Number of enclosures Total average watts (24 hrs) kWh/year (projected)

MSA60

84

8

672

5,887

The HP MSA70 storage solution, which consisted of 192 spindles in eight enclosures, consumed 952 average watts. This data is summarized in Table 10. The projected annual power consumption for this storage solution was 8,340 kWh/year.

noteNote:
The actual test environment consisted of four MSA60s and four MSA70s. For comparison, we calculated power consumption as if the complete solution used MSA70s.

Table 10 Power utilization for the native Exchange server environment using MSA70s

Role Average watts (24 hrs) Number of Enclosures Total average watts (24 hrs) kWh/year (projected)

MSA70

119

8

952

8340

noteNote:
Because identical storage solutions were used for both native and virtual Exchange environments, there was no difference in storage-related power consumption between native and virtual environments in this study.

Return to top

After these tests were finished, the data was analyzed and compared. The native Exchange 2007 servers, without accounting for storage utilization, consumed 1,962 average watts during the 24-hour test period. After the eight native Exchange 2007 servers were converted into eight virtual Exchange 2007 servers running on two physical Hyper-V root servers, the two root servers consumed 982 average watts during the 24-hour test period. This virtualization scenario resulted in a 50 percent reduction in power consumption as shown in the following figure. This translates into a projected savings of 8,582 kWh/year.

The MSA60 storage solution that was tested consumed 672 average watts during the 24-hour test period. If the storage power consumption is added to the server power consumption, 2,634 average watts were consumed by the native server environment, and 1,654 average watts were consumed by the virtual server environment. When the MSA60 storage solution was included, the virtualization scenario had a 37 percent reduction in power consumption.

The MSA70 storage solution consumed 948 average watts during the 24-hour test period. If storage power consumption is added to server power consumption, 2,910 average watts were consumed by the native server environment and 1,930 average watts were consumed by the virtual server environment. When the MSA70 storage solution was included, the virtualization scenario had a 34 percent reduction in power consumption.

Average Watts consumed - virtual and physical env

Return to top

This study examined power utilization of native and virtual Exchange 2007 environments in a scenario where physical servers were reduced from 8 to 2 but the total number of logical processors remained constant at 32. There was no processor core consolidation, and storage power utilization was not included. In this scenario, there was a 50 percent reduction in server power utilization and a projected savings of 8,582 kWh/year.

When you examine the power savings of the total solution, including servers and storage, you see a 37 percent reduction in power utilization when the MSA60 storage solution is deployed and a 34 percent reduction in power utilization when the MSA70 storage solution is deployed.

Notice that power utilization for the largest DAS solution (MSA70) accounts for less than 50 percent of the total power consumed by the virtual environment and less than 35 percent of the total power consumed by the native environment. When DAS solutions are used, storage is not the primary consumer of power.

importantImportant:
This study does not examine the effect of virtualization on reducing cooling requirements. In many environments, cooling can cause significant power consumption. Therefore, it should be included in any cost analysis. The Microsoft HyperGreen tool can be used to help estimate reduction in cooling costs for virtualization scenarios.

Although this study showed that virtualization of Exchange servers can generate power savings, power savings is not the only factor to consider when you evaluate whether virtualization of Exchange is appropriate for a specific environment. Adding virtualization to an Exchange environment can introduce additional complexity in several areas including backup, monitoring, and storage configurations. Virtualization of any Exchange solution should be carefully considered and planned, and virtualization scenarios should be tested with server loads that mimic the loads in the actual production environment. We recommend that you review the references in the "Additional Information" section before you consider implementation of Exchange on virtualized hardware.

Return to top

Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments

Should You Virtualize Exchange 2007 SP1?

The Microsoft HyperGreen tool

Windows Server Virtualization Validation Program

Windows Server Hyper-V

Hyper-V Planning and Deployment Guide

Microsoft Assessment and Planning Toolkit

Hyper-V Getting Started Guide

Performance Tuning Guidelines for Windows Server 2008 (Virtualization Section)

Hyper-V Performance FAQ

Monitoring Hyper-V Performance

Microsoft System Center Virtual Machine Manager

Note   The following performance results are provided for information and should not be used for sizing guidance or to make performance recommendations for virtualizing Exchange environments. For more information, see Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments.

Table A1 Comparison of key performance metrics for native and virtual Client Access servers

Counter Native Virtual

MSExchange OWA\Current Unique Users

2,000

2,000

MSExchange OWA\Average Response Time

29.5

29.0

MSExchange OWA\Requests/sec

55.7

53.1

MSExchange ActiveSync\Average Request Time

93

111

MSExchange ActiveSync\Requests/sec

14.6

15.2

% Processor Time (4 processor cores)

36%

40%*

Hyper-V Hypervisor Logical Processor\ %Guest Run Time

Not applicable

9%

*The processor was measured in the guest operating system. This value is inaccurate when it is measured in a Hyper-V guest operating system. See Table A4 for Hyper-V host data.

noteNote:
Values are an average of the two client access servers over the 10-hour workday period.

Return to top

Table A2 Comparison of key performance metrics for native and virtual Hub Transport servers

Counter Native Virtual

Logical Disk(*)\Average Disk sec/read

0.002

0.008*

Logical Disk(*)\Average Disk sec/write

0.001

0.013*

MSExchange Transport SMTP Receive\Messages received/sec

5.5

5.2

% Processor Time (4 processor cores)

29%

32%**

Hyper-V Hypervisor Logical Processor\ %Guest Run Time

Not applicable

7%

*These counter results may be higher than in the native environment because they were in a .vhd file. This file was located on a disk that was shared with other guest virtual machine vhds (virtual hard drives).

**The processor was measured in the guest operating system. This value was shown to be inaccurate when measured in a Hyper-V guest operating system. See Table A4 for Hyper-V host data.

noteNote:
Values are an average of the two Hub Transport servers over the 10-hour workday period.

Return to top

Table A3 Comparison of key performance metrics for native and virtual Mailbox servers

Counter Native Virtual

Disk Transfers/sec

1,717

1,752

IOPS/user

0.43

0.44

Logical Disk(*)\Average Disk sec/read

0.007

0.008

Logical Disk(*)\Average Disk sec/write

0.001

0.001

Disk Read Bytes/sec

11,827,000

12,666,000

Disk Write Bytes/sec

15,325,000

15,333,000

MSExchangeIS\RPC Average Latency

2.5

5.8*

MSExchangeIS\RPC Requests

4.6

8.3

MSExchangeIS Client\RPC Operations/sec (total)

3,175

2,934

MSExchangeIS Client\RPC Operations/sec (EAS)

685

545

MSExchangeIS Client\RPC Operations/sec (OWA)

474

357

MSExchangeIS Client\RPC Operations/sec (MAPI)

845

846

MSExchangeIS Client\RPC Operations/sec (CI)

343

314

MSExchangeIS Client\RPC Operations/sec (Transport)

714

695

% Processor Time (4 processor cores)

48%

47%*

Hyper-V Hypervisor Logical Processor\ %Guest Run Time

Not applicable

11%

*RPC Average Latency is inaccurate when it is measured in a Hyper-V guest operating system.

**The processor was measured in the guest operating system. This value has been shown to be inaccurate when measured in a Hyper-V guest operating system (see Table A4).

noteNote:
Values are an average of the four mailbox servers over the 10- hour workday period.

Table A4 Key performance metrics for the Hyper-V Root Servers

Counter Hyper-V root

Hyper-V Logical Processor Count

16

Hyper-V Hypervisor Logical Processor\ %Guest Run Time

37%

Hyper-V Hypervisor Logical Processor\ %Hypervisor Run Time

8%

Hyper-V Hypervisor Logical Processor\ %Idle Run Time

55%

Hyper-V Hypervisor Logical Processor\ %Total Run Time

45%

Hyper-V Hypervisor Logical Processor \Context Switches / sec

7,102,141

noteNote:
Values are an average from the two Hyper-V host servers over the 10-hour workday period.

Return to top

 
Did you find this helpful?
(1500 characters remaining)
Thank you for your feedback

Community Additions

ADD
Show:
© 2014 Microsoft. All rights reserved.