Running Exchange 2007 SP1 in a guest virtual machine does not change the Exchange Server design requirements from an application perspective. The Exchange Server guest virtual machine must still be sized appropriately to handle the workload. You take the same approach to sizing a virtualized Exchange server that you would take for sizing a non-virtualized Exchange server. Mailbox, Client Access, and transport server roles must still be designed for performance, capacity, and reliability. In addition, they must be allocated resources that are sufficient to handle the load on the system, based on the usage profiles of the system. For details and guidance about sizing Exchange server roles, see:

• Planning Processor Configurations
  
• Planning Memory Configurations
  
• Planning Storage Configurations
  
• Sizing Client Access Servers
  
• Mailbox Server Storage Design
  
• Transport Server Storage Design
  

 Dynamic Memory Allocation Considerations

Many hypervisors include features to dynamically adjust the amount of RAM that is available to one or more virtual machines. This functionality lets the hypervisor allocate RAM to virtual machines based on the current perceived RAM requirements of the particular virtual machines.

Generally, this functionality is appropriate for virtual machine workloads that use a lot of memory for brief periods and then resume typical operations. In this scenario, the hypervisor can allocate memory to meet the needs of the particular workload, and then retrieve the memory for other virtual machines. However, this functionality may not be suitable for workloads that are designed to use a particular memory pool on an ongoing basis.

Many of the performance improvements in recent versions of Exchange are based on the efficient use of an appropriately-sized RAM allocation. This is particularly true of improvements that are related to reductions in I/O operations. The performance optimizations rely on Exchange caching data in RAM. When RAM is dynamically reduced, the expected performance of the system cannot be achieved. In this scenario, Exchange may exhibit reduced performance, or end-users may experience reduced performance when connecting to Exchange. Therefore, for virtual machines that are running Exchange in a production environment, it is best to turn off memory oversubscription or dynamic memory allocation. Instead, configure a static memory size that is based on the appropriate values for Exchange 2007.

For more information memory considerations, see Planning Memory Configurations. For more information about dynamic memory allocation, see the "Application Considerations" section of the Hyper-V team whitepaper, Implementing and Configuring Dynamic Memory.

Following are considerations for Exchange guest virtual machines:

• When calculating the total number of allocated virtual processors, you must include the requirements of the root operating system. Many hardware virtualization products (for example, Hyper-V) allocate virtual processors to the root machine's operating system. The number of available virtual processors assigned to the root machine will vary, based on the number of physical processors and cores in the root machine and other configuration settings. In the case of Hyper-V, the number of virtual processors assigned to the root machine will equal the number of physical processor cores in the root machine. This number often represents a value that is larger than is generally required by most Exchange configurations.
  
• When calculating the total number of virtual processors required by the root machine, you must also account for both I/O and operating system requirements. In most cases, the equivalent number of virtual processors required in the root operating system for a system hosting Exchange virtual machines is 2. This value should be used as a baseline for the root operating system virtual processor when calculating the overall ratio of physical cores to virtual processors. If performance monitoring of the root operating system indicates you are consuming more processor utilization than the equivalent of 2 processors, you should reduce the count of virtual processors assigned to guest virtual machines accordingly and verify that the overall virtual processor-to-physical core ratio is no greater than 2:1.
  
• The Exchange server guest machine's storage and network design requires additional considerations for the root machine, specifically, the impact to the CPUs on the root machine. In some hardware virtualization environments (such as Hyper-V), all I/O requests that are made by guest virtual machines are serviced through the root machine. In these environments, we recommend that no other I/O intensive applications (for example, Microsoft SQL Server) be deployed on guest machines that are hosted on the same root machine as Exchange server guest machines.
  
• The addition of a virtualization layer to an Exchange server (where Exchange is running in a guest virtual machine) means that there are additional components that must be monitored for performance and availability. Additional considerations when monitoring Exchange in a virtual environment are:
  
  • CPU cycles inside a guest virtual machine occur relative to CPU time slices on the root machine. This behavior causes the values for CPU-related performance counters inside a guest virtual machine to be different from the values reported by the root machine. However, both of the values that are reported by each system are correct because they are based on the system's point-of-view and the way in which processor resources are shared among the root and guest machines. For information about this issue in a Hyper-V environment, see Hyper-V: Clocks lie... which performance counters can you trust?
    

    Note:
    The content of each blog and its URL are subject to change without notice. The content within each blog is provided "AS IS" with no warranties, and confers no rights. Use of included script samples or code is subject to the terms specified in the Microsoft Terms of Use.

  • Exchange-specific performance counters are available only in the guest machine. The root machine only publishes performance data for resources it is using directly and counters specific to the host environment (for example, Hyper-V's performance counters). The root machine does not publish any Exchange-specific performance data.
    
• Guest virtual machines on some hypervisors will exhibit processor core scalability trends that are different from processor scalability trends exhibited by physical machines. It is important that you thoroughly stress test the configured guest virtual machine prior to putting it into production.
  
• Some hypervisors include resource control features that enable you to balance resources among guest machines. For example, in a Hyper-V environment, you can specify a percentage of processor resources that are reserved for each guest machine. This is referred to as the virtual machine reserve. You can also specify a maximum percentage of processor resources that can be used by each guest machine. This is referred to as the virtual machine limit. In addition, you can assign a relative weight to each guest machine to dictate how resources should be allocated when multiple guest machines are running and competing for resources. In most environments, the hypervisor's resource control settings will not need to be modified from their default settings. We recommend that you check with your hypervisor manufacturer for configuration and tuning information.
  
• Supporting large mailboxes (for example, 1 GB and larger) requires the use of cluster continuous replication or hardware-based VSS solutions. Using hardware-based VSS is not possible in a hardware virtualization environment.
  

Exchange 2007 includes a number of high availability and disaster recovery features, such as local continuous replication (LCR), cluster continuous replication (CCR), standby continuous replication (SCR), and single copy clusters (SCC). All four configurations are supported in a virtualized environment.

Some hardware virtualization software includes features that support the clustering or portability of guest virtual machines across multiple physical root machines. For example, Hyper-V includes a clustered solution called quick migration, which combines Hyper-V host machines with Windows failover clustering. For more details about quick migration, you can download the Quick Migration with Hyper-V White Paper. With the Windows Server 2008 Enterprise and Windows Server 2008 Datacenter operating systems, you can run each server that provides client services as a guest virtual machine on a physical server and configure the physical server as a node in a failover cluster (a group of connected computers that work together to provide redundancy for services). In this configuration, other physical servers in the cluster are ready to support the guest virtual machines when needed through quick migration. The impact of quick migration on a guest machine depends on the nature of the outage:

• Scheduled Outages   A scheduled outage occurs when the administrator manually moves clustered resources to another node in the cluster. In this scenario, the guest machine state is suspended and saved, the resources are transferred to the specified node, and the guest machine is resumed from the saved state on the specified node. From an Exchange perspective, the server generally only experiences dropped TCP connections. Clients will experience an interruption in service during the migration process: Microsoft Office Outlook users in Cached Exchange Mode, as well as Exchange ActiveSync clients, will briefly be offline, and Outlook users in Online Mode, as well as Office Outlook Web Access, POP3, and IMAP4 users will be unable to access their mailboxes during the migration process. The length of the outage will depend on how long it takes to suspend, move, and resume the virtual machine and is heavily influenced by storage connectivity and the memory size of the virtual machine.
  
• Unscheduled Outages   An unscheduled outage occurs when a failure affects an active node to the degree of triggering cluster failover policies. For example, if the active node loses power or experiences a catastrophic software or hardware failure. In this scenario, the guest machine experiences an unexpected power loss. On an Exchange guest machine that hosts the Mailbox server role, the unexpected shutdown will put the databases in a dirty shutdown state. When the Exchange guest is restarted, Exchange will perform its built-in crash recovery processes and replay all log files for all databases. The amount of time it takes for recovery to complete depends on how many log files must be replayed; all log files generated after the checkpoint must be replayed. Generally, you can expect to achieve a replay rate of at least two log files per second.
  

 Choosing a High Availability Solution for a Virtualized Exchange Server

We recommend using the built-in Exchange Server high availability solutions for virtualized Exchange servers instead of hypervisor-provided clustering or portability solutions (such as Hyper-V's quick migration feature). The features found in Exchange Server (in particular, cluster continuous replication (CCR)) provide greater benefits than those found in hypervisor solutions that move virtual machines between physical root machines.

The built-in data replication mechanism in Exchange (continuous replication) can be combined with Windows failover clustering in a configuration known as CCR. CCR can be deployed in a hardware virtualization environment, thereby providing you with a service and data availability solution for Exchange 2007 in a virtualized environment. In a virtualized CCR (or SCC) environment, each guest machine that is a node in the cluster must be hosted on separate physical root machines to provide true redundancy and high availability.

Note:
It is supported to deploy CCR or SCC using a combination of physical nodes and virtual nodes. As with all Exchange high availability configurations, you must ensure that all nodes are sized appropriately to handle the full workload during scheduled or unscheduled outages.

We do not recommend using hypervisor-based virtual machine migration (such as Hyper-V's quick migration) for virtualized Exchange servers. In a virtual machine migration configuration, an unscheduled outage can result in data loss. In a CCR environment, this type of data loss is largely mitigated by a feature called transport dumpster. The transport dumpster takes advantage of the redundancy in the environment to reclaim some of the data affected by the failover. For more information, see Cluster Continuous Replication.

The differences between virtual machine migration solutions and CCR (when deployed in a hardware virtualization environment) are listed in the following table:

Virtual machine migration vs. cluster continuous replication

	Virtual machine migration	Cluster continuous replication
Operating system heartbeat detection	Yes	Yes
Exchange server heartbeat detection	No	Yes
Copies of Exchange data	1	2
Requires shared storage	Yes	No
Supports Exchange-aware backup from passive node	No	Yes

 Backup and Restore Considerations

Exchange Server has significant I/O requirements. When deploying large Exchange servers in guest virtual machines, we recommend you use pass-through disks for data storage. The current implementation of VSS in Hyper-V does not support root-based backups for pass-through disks or iSCSI disks that are connected to an iSCSI initiator inside the guest virtual machine. As a result, VSS backups of an Exchange guest virtual machine that are performed within the root machine are not supported for pass-through or iSCSI disks connected within the guest virtual machine.

To perform supported backups of a virtualized Exchange server using either of these storage types, you must perform backups from within the guest virtual machine. You can use backup software that supports the ESE streaming backup APIs or an Exchange-aware software-based VSS solution (for example, Microsoft System Center Data Protection Manager).

VSS backup of an Exchange guest virtual machine from within the root machine is supported when Hyper-V virtual hard disks (VHDs) are used.

Some storage solutions also include storage vendor-supported methods for directly taking hardware-based VSS backups of the storage volumes. Support for these backup methods is provided by the storage vendor and not by Microsoft.

We recommend that you use separate LUNs that are protected using redundant array of independent disks (RAID) arrays for the host operating system, each guest operating system disk, and all virtual machine storage. LUNs for databases and log files should be isolated in accordance with Exchange 2007 storage best practices. All Exchange 2007 storage requirements and best practices apply to Exchange servers running in a hardware virtualization environment. For more information about storage requirements, recommendations, and best practices for Exchange 2007, see Planning Storage Configurations.

The following figure illustrates the storage configuration of Exchange 2007 in a Hyper-V environment.

Storage configuration of Exchange 2007 in a Hyper-V environment

 Guest Virtual Machine Storage Requirements

The operating system for an Exchange guest machine must use a disk that is at least 15 GB plus the size of the virtual memory that is allocated to the guest machine. This requirement is necessary to account for the operating system and paging file disk requirements. For example, if the guest machine is allocated 16 GB of memory, the minimum disk space needed for the guest operating system disk is 31 GB.

In addition, it's possible that guest virtual machines may be prevented from directly communicating with fibre channel or SCSI host bus adapters (HBAs) installed in the root machine. In this event, you must configure the adapters in the root machine's operating system and present the LUNs to guest virtual machines as either a virtual disk or as a pass-through disk.

 Root Machine Storage Requirements

Each root machine has minimum disk space requirements that must be met:

• Root machines in some hardware virtualization applications may require storage space for an operating system and its components. For example, when running Windows Server 2008 with Hyper-V, you will need a minimum of 10 GB to meet the Windows Server 2008 System Requirements for the operating system. Additional storage space will also be required to support the operating system's paging file, management software, and crash recovery (dump) files.
  
• Some hypervisors maintain files on the root machine that are unique to each guest virtual machine. For example, in a Hyper-V environment, a temporary memory storage file (BIN file) is created and maintained for each guest machine. The size of each BIN file is equal to the amount of memory allocated to the guest machine. In addition, other files may also be created and maintained on the host machine for each guest machine.
  

 Exchange Server Storage Requirements and Recommendations

Following are the requirements and recommendations for storage connected to a virtualized Exchange server:

• Each Exchange Server guest machine must be allocated sufficient storage space on the root machine for the disk that contains the guest's operating system, any temporary memory storage files in use, and related virtual machine files that are hosted on the host machine. In addition, for each Exchange guest machine, you must also allocate sufficient storage for the message queues on Hub Transport and Edge Transport servers and sufficient storage for the databases and log files on Mailbox servers.
  
• Storage used by Exchange should be hosted in disk spindles that are separate from the storage that is hosting the guest virtual machine's operating system.
  
• Virtual disks may not perform as well as other disk types. You should consult the performance and scalability documentation from your hypervisor manufacturer to understand how Exchange I/O may be affected by the use of different storage options.
  
• We recommend using SCSI pass-through storage to host transport and mailbox databases and transaction log files. Although using pass-through disks limits the portability of virtual machines, this configuration has shown to provide the best performance of all storage options for a virtualized Exchange server.
  
• When using iSCSI storage, the best performance is achieved when the iSCSI Initiator component is configured on the host machine, and the disks are presented to the guest machine as pass-through disks. We recommend using Gigabit Ethernet speeds or faster for the iSCSI storage and isolating the iSCSI storage network from all other traffic. We also recommend using dedicated physical network interface cards for iSCSI network traffic. In a Hyper-V environment, we further recommend that the dedicated physical iSCSI network cards be configured to use jumbo frames and not be bound to any Virtual Network Switch.
  
• Configuring iSCSI storage to use an iSCSI initiator inside an Exchange guest virtual machine is supported. However, there will be reduced performance in this configuration because the network stack inside a virtual machine is not as full-featured as a non-virtualized network stack (for example, a virtual network stack does not support jumbo frames). However, because the iSCSI storage is connected directly to the guest's iSCSI initiator and not configured as pass-through disks, the virtual machine has greater portability.
  

We recommend specific network configurations when running Exchange 2007 in a hardware virtualization environment. These configurations are based on whether Exchange is deployed for high availability.

For environments that are not deployed for high availability (for example, no CCR, no SCC, no Quick Migration), we recommend that you follow the planning and deployment guidance provided by your hypervisor vendor. For example, in the case of Hyper-V, we recommend that you follow the guidance set forth in the Hyper-V Planning and Deployment Guide and the Microsoft Hyper-V Server 2008 Configuration Guide.

For environments that are deployed for high availability (for example, CCR or SCC), we recommend having at least two physical network interface cards (NICs) in the root machine. One NIC should be dedicated to the hypervisor root machine, and the other NIC should be dedicated for the guest virtual machines. Additional separate NICs should be used for any iSCSI storage that is in use by the root or guest machines.

Note:

If you are deploying Client Access or Hub Transport guest machines in Hyper-V that are also configured for high availability by using Network Load Balancing (NLB), then you must install the hotfix from Microsoft Knowledge Base article 953828, "The NLB host does not converge as expected on Windows Server 2008 Hyper-V virtual machines". For more information about how to install and configure NLB, see the Network Load Balancing Deployment Guide.

When you plan to deploy Exchange Server 2003 SP2 in a virtualized environment, the same performance and scalability aspects that are described in the Exchange Server 2003 Performance and Scalability Guide apply when you size each virtual machine for Exchange Server 2003.

However, some factors directly affect the performance and scalability of Exchange Server 2003 when it is running on Virtual Server 2005 R2. These factors should be considered when you size both the host configurations and the guest configurations.

Following are the factors to consider about the virtual machine configuration:

• Each virtual machine can have only one CPU. This limits the processing power of the virtualized Exchange installation. The server should be sized in such a way that a single CPU can handle the estimated load on the server. Also, the number of virtual machines that are running at the same time on the host computer will affect the overall performance of the whole system.
  
• When you size the disk capacity of the virtual machine, the time that is required to perform a full online backup of the Exchange Server data over the network should be considered. Consider adding a dedicated virtual network adaptor for Exchange Server backups.
  

  Note:
  Although you can perform an offline backup of the virtual hard disk (.VHD) files at the host level, this does not replace the need for performing a regular Exchange Server backup. For more information about Exchange Server backup and restore processes, see the Exchange Server 2003 Disaster Recovery Operations Guide.

• Create separate fixed-size virtual disks for Exchange Server databases and log files and store them on separate physical drives on the host server.
  
• Exchange Server performance should be validated before production by using the Exchange Server 2003 Performance Tools. For more information about these tools, see the Exchange Server 2003 Performance and Scalability Guide.
  

Following are the factors to consider about the host configuration:

• Make sure that the server that is running Virtual Server 2005 R2 is sized correctly to handle the number of virtual machines that you plan to deploy. This estimate should include CPU, memory, network adaptors, and disk configuration.
  
• Use a hard disk solution that enables fast access. You can use a SCSI hard disk, a redundant array of independent disks (RAID), or storage area network (SAN) to store the .VHD files that are used by Exchange Server data.
  
• If an antivirus program is installed on the host, the antivirus program should be configured not to scan .VHD files.