A typical back-end mailbox server will consume between 3 gigabytes (GB) and 4 GB of physical memory. Exchange Server does not utilize instancing, Physical Address Extension (PAE), or Address Windowing Extensions (AWE). Therefore, 4 GB of random access memory (RAM) is the maximum amount of memory that an Exchange Server computer can efficiently use.

Note that running with PAE enabled is supported. Although Exchange Server cannot take advantage of more than 4 GB of memory, other features may rely on the PAE kernel.

If your Exchange Server 2003 server has 1 GB or more of memory installed, and if the computer stores mailboxes or public folders, make sure that you add the /3GB switch to the Boot.ini file on the server. By default, the Windows Server 2003 and Windows 2000 Advanced Server operating systems reserve 2 GB of virtual address space for system usage and another 2 GB for the user address space. Each process is allocated this range of virtual addresses at startup. Typically, the actual memory usage (the working set) of a process is much less than the address space allocated to that process. On a computer that is running Exchange Server 2003 with 1 GB or more of memory, you should modify the Windows Server 2003 and Windows 2000 Advanced Server operating systems so that 3 GB of user address space is available to each process. You can do this by using the /3GB switch in the Boot.ini file.

Remember that while /3GB is beneficial for the smooth running of Exchange, kernel memory resources are squeezed into 1 GB of virtual address space. This means that the maximum paged pool and nonpaged pool sizes are significantly lower. In addition, free system page table entries (PTEs), which map virtual addresses to physical memory, reduce from approximately 200,000 to 15,000. If a server runs low on free system PTEs, instability, including stop errors, can occur. Both Windows Server 2003 and Windows 2000 Server offer mechanisms to increase the number of free system PTEs.

For a Windows 2000 Advanced Server-based computer, you should configure the SystemPages registry in the following registry subkey:

\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\SystemPages

Set the SystemPages registry entry to a DWORD value of 0x7918, which is 31000 in decimal notation. In some articles, you may see a recommendation to use 0xFFFFFFFF. Although the latter is a valid setting and can be used to gain the maximum number of free system PTEs, you must remember that memory is taken from the paged pool to bolster the number of free system PTEs. In the scale-up environments that you see today, paged pool resources are also critical to the running of an Exchange server. Therefore, you need to carefully balance the resources of all kernel memory objects. This same registry change is not required, nor is it recommended, on Windows Server 2003-based servers, because this functionality is achieved by using the /USERVA switch that is outlined in a later section.

Do not set the /3GB switch if you are running Windows 2000 Server. This memory tuning switch is not supported on Windows 2000 Server. Although the server will not generate an error message if you do this, the result of setting this switch is that a false memory address space will exist. In cases where a process tries to access this higher address space, a stop error occurs, and the server stops responding.

The /USERVA switch is new to Windows Server 2003 and provides better granularity for splitting virtual address space between user mode and kernel mode. This behavior lets you scale the server to a greater number of users without the risk of exhausting kernel resources. By using /USERVA=3030, an additional 42 megabytes (MB) of memory is allocated to the kernel for PTEs. However, this value may need more tuning. You can monitor the PTE consumption by using Performance Monitor. Monitor the Free System Page Table Entries counter. This counter can be found under the Memory object. If Performance Monitor reports a value of 8,000 or less, the /USERVA value of 3,030 must be reduced to avoid system instability.

Note:

Microsoft Product Support Services strongly recommends using a range of memory that is within the range of 2,800 to 3,030 for the /USERVA switch. This range is wide enough to provide a large enough pool of system PTEs for all currently observed issues. Typically, a setting of /USERVA=2800 provides close to the maximum available number of system PTEs that are possible. Remember that when you reduce the value for /USERVA, the large address space benefit to the Exchange store process is reduced. For maximum performance of both Exchange and the underlying operating system, careful tuning and monitoring is necessary.

The following table shows an overview of how and when the /3GB and /USERVA switches should be used. This applies to Exchange Server 2003 with Service Pack 1 (SP1) and Windows Server 2003 SP1.

* Envelope journaling, MTA, and third-party connectors send gateway and bridgehead e-mail through the Store.exe process so the system benefits from /3GB.

** /3GB and /USERVA=3030 switches are not used on Exchange servers with less than 1 GB RAM.

When working with the /3GB and /USERVA switches, consider the following:

• For additional details about configuring Windows memory for Exchange Server 2003, see Microsoft Knowledge Base article 823440, "Use of the /3GB switch in Exchange Server 2003 on a Windows Server 2003-based system."
  
• Windows may add or require additional Boot.ini configuration switches.
  
• The previously listed settings are applicable to all versions of Exchange Server 2003 and the Windows Server 2003 operating system.
  

PAE is the added ability of the IA32 processor to address more than 4 GB of physical memory. Microsoft Windows Server 2003, Enterprise Edition, Microsoft Windows Server 2003, Datacenter Edition, Microsoft Windows 2000 Advanced Server, and Microsoft Windows 2000 Datacenter Server, can use PAE to take advantage of physical memory beyond 4 GB. To enable PAE, use the /PAE switch in the Boot.ini file. Newer servers that support hot-add memory and run on the Windows Server 2003 operating system will automatically load the PAE kernel. Additionally, the PAE kernel will be loaded on servers that support hardware-enforced data execution prevention (DEP) and run on the Windows Server 2003 SP1 operating system. This has been a source of confusion in the past as servers were identified as using the PAE kernel even when /PAE was not set in the Boot.ini.

When more physical memory is used on the system, the process of paging memory to the disk increases dramatically, and performance may be negatively impacted. The Windows Server 2003 and Windows 2000 Server memory managers use PAE to provide more physical memory to the operating system. This reduces the need to swap the memory in and out of the page file and results in increased performance. The program itself is not aware of the actual memory size. All the memory management and allocation of the PAE memory is handled by the memory manager independently of the running programs.

The preceding information is valid for programs that run when the /3GB switch is used. A program that requests a large amount of memory is more likely to have more of its memory remain in physical memory rather than be paged out. This increases the performance of programs that are capable of using the /3GB switch. The exception is when the /3GB switch is used with the /PAE switch. In this case, the operating system does not use any memory in excess of 16 GB. This behavior is caused by kernel virtual memory space considerations. Thus, if the system restarts with the /3GB entry in the Boot.ini file, and the system has more than 16 GB of physical memory, the additional physical RAM is not used by the operating system. Restarting the computer without the /3GB switch enables the use of all the physical memory.

Note:
Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition automatically enable PAE only if the server is using hot-add memory devices. In this case, you do not have to use the /PAE switch on a system that is configured to use hot-add memory devices. In all other cases, you must use the /PAE switch in the Boot.ini file to take advantage of memory over 4 GB.

As you add more memory to a server, the requirements for kernel resources are increased. For example, system PTEs track virtual to physical memory address locations. The more physical memory you have, the more system PTEs will be used, thus resulting in less free system PTEs. Newer servers come preconfigured with 6 GB or 8 GB of memory. As a best practice, for servers dedicated to Exchange, you should consider removing memory over 4 GB or limiting the memory recognized by the operating system with the /MAXMEM switch in the Boot.ini file. If you do not do this, not only will the memory be wasted, but it could lead to further instability due to kernel memory resource constraints.

To determine whether the PAE kernel is loaded when the computer starts, you can view the following registry subkey:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PhysicalAddressExtension

If the PhysicalAddressExtension DWORD value is set to 00000001, the PAE kernel is loaded when the computer starts. This key is set by the operating system at startup. Users cannot configure this parameter.

Until recently, it was a best practice to disable PAEs with the /NOPAE switch on Exchange servers where the /3GB switch was set. This recommendation existed because Exchange Server 2003 or Exchange 2000 Server had not been tested with the PAE kernel, nor was the PAE kernel necessary on an Exchange server. New hardware and software capabilities now require the PAE kernel to be loaded. For example, servers with Lindenhurst or Twincastle chipsets require the PAE kernel to be loaded, otherwise not all installed memory will be recognized. In another example, Windows Server 2003 SP1 supports hardware-enforced DEP. This functionality cannot be enabled if the PAE kernel is not loaded.

As a result of testing by the Exchange Server product team, it is now a best practice to allow the PAE kernel to load on an Exchange server. Therefore, the /NOPAE switch is no longer necessary unless you are troubleshooting a specific issue where the PAE kernel is suspected as a contributory factor. On some servers running Windows Server 2003 SP1, the /NOPAE switch may be ineffective because hardware-enforced DEP, which is enabled with the /NoExecute switch, will override this instruction to disable PAE. If required, the /Execute switch can be used to disable the PAE kernel. For a description, see Microsoft Knowledge Base article 900524, "How to prevent the PAE kernel from loading in Windows Server 2003 with Service Pack 1 or in Windows XP with Service Pack 2."

There are several other settings that can be changed on Exchange Server 2003 servers to optimize memory utilization. Some of the settings are the following:

• Maximizing virtual address space by changing the HeapDeCommitFreeBlockThreshold registry value. This registry setting is critical to the health and stability of back-end Exchange servers.
  
• Changing the Store Database Cache size, also known as the Extensible Storage Engine (ESE) buffer size, by making Active Directory® directory service changes.
  

For more information about memory-related settings, see Microsoft Knowledge Base article 815372, "How to optimize memory usage in Exchange Server 2003."

Event 9665, which is a warning event, might be logged in the application log of an Exchange Server 2003 server. The event wording will be as follows:

Event Type: Warning

Event Source: MSExchangeIS

Event Category: General

Event ID: 9665

Description:

The memory settings for this server are not optimal for Exchange.

This is a part of new functionality built into Exchange Server 2003, which checks for several known memory configuration settings at startup.

If Event 9665 is logged, for more information about troubleshooting, see Microsoft Knowledge Base article 815372, "How to optimize memory usage in Exchange Server 2003."