The first metric to understand is mailbox size. Knowing the amount of data that an end user is allowed to store in his or her mailbox allows you to determine how many users can be housed on the server. Although final mailbox sizes and quotas change, having a goal is the first step in determining your needed capacity. For example, if you have 5,000 users on a server with a 250 megabyte (MB) mailbox quota, you need at least 1.25 terabytes of disk space. If a hard limit is not set on mailbox quotas, it will be difficult to estimate how much capacity you will need.

The database size on the physical disk is not just the number of users multiplied by the user quota. When the majority of users are not near their mailbox quota, the databases will consume less space, and white space is not a capacity concern. The database itself will always have free pages, or white space, spread throughout. During online maintenance, items marked for removal from the database are removed, which frees these pages. The percentage of white space is constantly changing with the highest percentage immediately after online maintenance and the lowest percentage immediately before online maintenance.

The size of white space in the database can be approximated by the amount of mail sent and received by the users with mailboxes in the database. For example, if you have 100 2-GB mailboxes (total of 200 GB) in a database where users send and receive an average of 10 MB of mail per day, the white space is approximately 1 GB (100 mailboxes × 10 MB per mailbox).

White space can grow beyond this approximation if online maintenance is not able to complete a full pass. It is important that your operational activities include enough time for online maintenance to run each night, so that a full pass can complete within one week or less.

Each database has a dumpster that stores soft-deleted items. By default, items are stored for 14 days in Microsoft Exchange Server 2007. These include items that have been removed from the Deleted Items folder. By default, compared with Exchange Server 2003, Exchange 2007 increases the overhead consumed by the database dumpster because deleted items are now stored for twice as long. The actual amount in the dumpster will depend on the size of each item and your organization's specific retention settings.

After the retention period has passed, these items will be removed from the database during an online maintenance cycle. Eventually, a steady state will be reached where your dumpster size will be equivalent to two weeks of incoming/outgoing mail, as a percentage of your database size. The exact percentage depends on the amount of mail deleted and on individual mailbox sizes.

The dumpster adds a percentage of overhead to the database dependent upon the mailbox size and the message delivery rate for that mailbox. For example, with a constant message delivery rate of 52 MB per week, a 250-MB very heavy profile mailbox would store approximately 104 MB in the dumpster, which adds 41 percent overhead. A 1-GB mailbox storing the same 104 MB in the dumpster adds 10 percent overhead.

Over time, user mailboxes will reach the mailbox quota, so an amount of mail equivalent to the incoming mail will need to be deleted to remain under the mailbox quota. This means that the dumpster will increase to a maximum size equivalent to two weeks of incoming/outgoing mail. If the majority of users have not reached the mailbox quota, only some of the incoming/outgoing mail will be deleted, so the growth will be split between the dumpster and the increase in mailbox size. For example, a 250-MB very heavy message profile mailbox that receives 52 MB of mail per week (with an average message size of 50 kilobytes (KB)) would result in 104 MB in the dumpster (41 percent), and 7.3 MB in white space, for a total mailbox size of 360 MB. Another example is a 2-GB very heavy message profile mailbox that receives 52 MB of mail per week, which results in 104 MB in the dumpster (5 percent) and 7.3 MB in white space, for a total mailbox size of 2.11 GB. Fifty 2-GB mailboxes in a storage group total 105.6 GB.

The following is a formula for database size using a 2-GB mailbox:

Mailbox Size = Mailbox Quota + White Space + (Weekly Incoming Mail × 2)

Mailbox Size = 2,048 MB + (7.3 MB) + (52 MB × 2)

2,159 MB = 2,048 MB + 7.3 MB + 104 MB (5 percent larger than the quota)

After you have determined the projected actual mailbox size, you can use that value to determine the maximum number of users per database. Take the projected mailbox size, and divide it by the maximum recommended database size. This will also help you determine how many databases you will need to handle the projected user count, assuming fully populated databases. Be aware that due to non-transactional input/output (I/O) or because of hardware limitations, you may have to modify the number of users placed on a single server. Some administrators will prefer to use more databases to further shrink the database size. This approach can assist with backup and restore windows at the cost of more complexity in managing more databases per server.

Content indexing creates an index, or catalog, that allows users to easily and quickly search through their mail items rather than manually search through the mailbox. Exchange 2007 creates an index that is about 5 percent of the total database size, which is placed on the same LUN as the database. An additional 5 percent capacity needs to be factored into the database LUN size for content indexing.

A database that needs to be repaired or compacted offline will need capacity equal to the size of the target database plus 10 percent. Whether you allocate enough space for a single database, a storage group, or a backup set, additional space will need to be available to perform these operations.

If you plan to use a recovery storage group as part of your disaster recovery plans, enough capacity will need to be available to handle all of the databases you want to be able to simultaneously restore on that server.

Many administrators perform streaming online backups to a disk target. If your backup and restore design involves backup to disk, enough capacity needs to be available on the server to house the backup. Depending on the backup type you use, this capacity can be as small as the database and logs to as large as the database and all logs since the last full backup.

Most enterprises that perform a nightly full backup will allocate the capacity of about three days of log files in a storage group on the transaction log LUN. This is done to prevent a backup failure from causing the log drive to fill, which would dismount the storage group.

Log LUN sizing is somewhat dependent on your backup and restore design. For example, if your design allows you to go back two weeks and replay all of the logs generated since then, you will need two weeks of log file space. If your backup design includes weekly full and daily differential backups, the log LUN needs to be larger than an entire week of log file space to allow both backup and replay during restore.

Moving mailboxes is a primary capacity factor for large mailbox deployments. Many large companies move a percentage of their users on a nightly or weekly basis to different databases, servers, or sites. If your organization does this, you may find it necessary to provide extra capacity to the log LUN to accommodate user migrations. Although the source server will log the record deletions, which are small, the target server must write all transferred data first to transaction logs. If you generate 10 GB of log files in one day, and keep a 3-day buffer of 30 GB, moving 50 2-GB mailboxes (100 GB) would fill your target log LUN and cause downtime. In cases such as these, you may have to allocate additional capacity for the log LUNs to accommodate your move mailbox practices.

For most deployments, we recommend that you add an overhead factor of 20 percent to the log size (after all other factors have been considered) when creating the log LUN to ensure necessary capacity exists in moments of unexpected log generation.

In previous versions of Exchange Server, one of the key metrics needed for sizing storage is the amount of database I/O per second (IOPS) consumed by each user. To measure your user IOPS, take the amount of I/O (both reads and writes) on the database LUN for a storage group, and divide that by the number of users in that storage group. For example, 1,000 users causing 1,000 I/Os on the database LUN means you have an IOPS of 1.0 per user.

If you are using a previous version of Exchange Server and have calculated your baseline IOPS, keep in mind that Exchange 2007 will affect your baseline in the following ways:

• The number of users on the server will affect the overall database cache per user.
  
• The amount of RAM influences how large your database cache can grow, and a larger database cache results in more cache read hits, thereby reducing your database read I/O.
  

The key is that knowing your IOPS on a particular server is not enough to plan an entire enterprise because the amount of RAM, number of users, and number of storage groups will likely be different on each server. After you have your actual IOPS numbers, always apply a 20 percent I/O overhead factor to your calculations to add some reserve. You do not want a poor user experience because activity is heavier than normal.

A 64-bit Windows Server operating system running the 64-bit version of Exchange 2007 substantially increases the virtual address space, and allows Exchange to increase its database cache, reduce database read I/O, and enable up to 50 databases per server.

The database read reduction depends on the amount of database cache available to the server and the user message profile. For guidance on memory and storage groups, see Planning Processor Configurations. Following the guidance in that topic can result in up to a 70 percent transactional I/O reduction over Exchange 2003. The amount of database cache per user is a key factor in the actual I/O reduction.

The following table demonstrates the increase in actual database cache per user when comparing the default 900 MB in Exchange 2003 versus 5 MB of database cache per user in Exchange 2007. It is this additional database cache that enables more read hits in cache, thus reducing database reads at the disk level.

Database cache sizes based on mailbox count

Unlike Cached Exchange Mode clients, all Online Mode client operations occur against the database. As a result, read I/O operations will increase against the database. Therefore, the following guidelines have been established if the majority of clients will operate in Online Mode:

• 250 MB Online Mode clients will increase database read operations by a factor of 1.5 when compared with Cached Exchange Mode clients. Below 250 MB, the impact is negligible.
  
• As mailbox size doubles, the database read IOPS will also double (assuming equal item distribution between key folders remains the same).
  

The following graph illustrates IOPS based on mailbox size.

Database read IOPS increases as mailbox size increases

Testing has also shown that increasing the database cache beyond 5 MB per mailbox will not significantly reduce the database read I/O requirements. The following graph depicts 2-GB mailboxes using Online Mode clients and the effect increasing the cache beyond 5 MB has on reducing the database read I/O requirements.

Database read IOPS decreases cache size per mailbox increases

As a result of this data, two recommendations can be made:

• Deploy cached mode clients where appropriate. See the "Item Count per Folder" section below for more information.
  
• Ensure that the I/O requirements are taken into consideration when designing the database storage.
  

For additional IOPS factors, such as third-party clients, see Optimizing Storage for Exchange Server 2003.

In Exchange 2007, messages are indexed as they are received, causing little disk I/O overhead. Searching against the index in Exchange 2007 is approximately 30 times faster than in Exchange 2003.

Messaging records management (MRM) is a new feature in Exchange 2007 that helps administrators and users manage their mailboxes. Policies can be set to move or delete mail that meets specific thresholds, such as age. MRM is a scheduled crawl that runs against the database in a synchronous read operation similar to backup and online maintenance. The disk cost of MRM depends upon the number of items requiring action (for example, delete or move).

We recommend that MRM not run at the same time as either backup or online maintenance. If you use continuous replication, you can offload Volume Shadow Copy Service (VSS) backups to the passive copy, allowing more time for online maintenance and MRM so that they do not affect one another or users.

Many actions are performed when the database runs online maintenance, such as permanently removing deleted items and performing online defragmentation of the database. Maintenance causes reads, while online defragmentation causes both reads and writes. The amount of time it takes to complete maintenance is proportional to the size of the database and can be a limiting factor on how large databases are allowed to grow. For more information about online maintenance, see Store Background Processes Part I.

Note:
The content of each blog and its URL are subject to change without notice. The content within each blog is provided "AS IS" with no warranties, and confers no rights. Use of included script samples or code is subject to the terms specified in the Microsoft Terms of Use.

Many different backup and restore methods are available to the administrator. The key metric with backup and restore is the throughput, or the number of megabytes per second that can be copied to and from your production LUNs. After you determine the throughput, you need to decide if it is sufficient to meet your backup and restore service level agreement (SLA). For example, if you need to be able to complete the backup within four hours, you may have to add more hardware to achieve it. Depending on your hardware configuration, there may be gains that can be achieved by changing the allocation unit size. This can help with both streaming online backups, and the Exchange Server Database Utilities (Eseutil.exe) integrity check that occurs during a VSS backup.

With 2,000 users on a server, moving from a 200-MB mailbox to a 2-GB mailbox increases the database size tenfold. Many administrators are not accustomed to having to deal with large amounts of data on a single server. Consider a server with 2,000 2-GB mailboxes. With the overhead described previously, this is more than 4 terabytes of data. Assuming you can achieve a backup rate of 175 GB per hour (48 MB per minute), it would take at least 23 hours to back up the server. An alternative for servers that do not use LCR or CCR might be to perform a full backup of one-seventh of the databases each day, and an incremental backup on the remainder, as illustrated in the following table.

Example of a weekly backup routine

As you can see from the preceding table, the total amount of data backed up nightly is approximately 650 GB, which could complete in 3.7 hours, assuming a rate of 175 GB per hour. Some solutions can achieve more or less throughput. However, large mailboxes may require different approaches.

With LCR and CCR, the passive copy is the first line of defense. You only restore from backup if both the active copy and the passive copy fail or are otherwise unavailable. Recovering multiple days of incremental logs can add to the length of time it takes to recover. For this reason, incremental backup is seldom used on a fast recovery solution, such as CCR or VSS clones. With a VSS clone, the recovery of the data is fast, and adding a little time to replay logs may be acceptable to keep backup times within the backup SLA.

With streaming backups, we recommend that you separate streaming I/O (source and target) so that multiple storage groups being backed up concurrently do not compete for the same disk resources. Whether the target is disk or tape, there will be a throughput limit on the physical disks and controllers unique to each hardware solution. It may be necessary to isolate some storage groups from each other to maximize the number of concurrent backup operations, and throughput to minimize the size of the backup window. The following table illustrates an example of two concurrent backups of 14 databases.

Example of a concurrent backup routine

You can run streaming backups concurrently, one from each LUN, if you isolate your storage group LUNs from each other, as illustrated in the preceding table. The backup jobs should complete on the first storage group on each LUN before the second storage group begins to back up, isolating the backup streams. Two streaming backup jobs on the same physical disks may not be twice as fast, but it should be faster than a single streaming backup job in terms of megabytes per second.

Exchange 2007 uses VSS, which is included in Windows Server 2003, to make volume shadow copies of databases and transaction log files. For more information about VSS, including both clone and snapshot techniques, see Best Practices for Using Volume Shadow Copy Service with Exchange Server 2003. New in Exchange 2007 is the ability to make VSS backups of the passive copy of storage groups running in an LCR or CCR environment. In these environments, taking a VSS snapshot from the passive copy removes the disk load on the active LUNs during both the checksum integrity phase of the backup, and the subsequent copy to tape or disk. It also frees up more time on the active LUNs to run online maintenance, MRM, and other tasks.

In many cases, the LUN that the operating system recognizes is abstracted from the physical hardware that is actually behind that disk. It has always been critical to separate transaction logs from the database at both the LUN and physical disk level for performance and recoverability purposes. On some storage arrays, mixing random and sequential I/O on the same physical disks can reduce performance. From the perspective of recovery, separating a storage group's transaction logs and databases makes sure that a catastrophic failure of a particular set of physical disks does not cause a loss of both database and transaction logs.

Exchange database I/O is random, and most storage subsystems benefit when the physical disks are performing the same workload. Many storage arrays use virtual storage so that many physical disks are first pooled into a group of disks, and then LUNs are created from the available space in that disk group and distributed equally across every physical disk. When not using continuous replication, it is acceptable for the physical disks that are backing up a storage group's database LUN to also back up other LUNs that house the databases for other storage groups and servers. Likewise, it is not critical to isolate each storage group's transaction log LUN onto separate physical spindles, even though the loss of sequential I/O will slightly affect performance. It is important to separate the log and database LUNs from the same storage group onto separate physical disks. It is not realistic to dedicate two or four 500-GB physical disks to a single storage group's transaction log LUN when you require 30 IOPS and 5 percent of the capacity.

Although there are many ways to design LUNs in Exchange 2007, we recommend the following two designs to limit complexity:

• Two LUNs per storage group
  
• Two LUNs per backup set
  

Two LUNs per Storage Group

Two LUNs per Backup Set

There are many cases, such as in multiple node single copy clusters (SCCs), where more LUNs are needed than there are available drive letters. In those cases, you must use volume mount points. Drive letters are a legacy MS-DOS operating system feature to recognize partitions or disks, and it is best to avoid using too many drive letters. It is much easier to place all transaction log and database LUNs on a volume mount point for ease of administration. If you have 20 storage groups, each with a database, it is difficult to remember which drive letter houses database 17. The following table illustrates an example of using volume mount points.

Example folder layout using volume mount points

In this example L: and P: are anchor LUNs, which house all the log and database LUNs respectively. Each folder on these drives is a volume mount point to a separate LUN.

When using hardware-based VSS, there are a few recommendations for placing Exchange data on the LUNs. For a hardware-based VSS solution, each transaction log LUN and database LUN should only house the files from the chosen backup set. If you want to restore a storage group without affecting any other storage group, you will need a separate transaction log LUN and database LUN for each storage group. If you are willing to take other databases and storage groups offline to restore a single database, you can place multiple storage groups on a single transaction log LUN and database LUN.

When using software-based VSS, particularly with large mailboxes and continuous replication, your backup is a two-step strategy. First, you take a VSS snapshot, and then you stream the flat files to disk or tape.

It is always important to place a storage group's transaction logs and databases on separate physical disks because doing so increases reliability. With continuous replication, it is also important to separate the active and passive LUNs on completely separate storage. With CCR and LCR, you want storage resiliency in the event of a catastrophic failure of the primary storage.

LUN Example

Move Mailbox

A larger maximum database size is possible when continuous replication is used. We recommend the following maximum database sizes for Exchange 2007:

• Databases hosted on a Mailbox server without continuous replication: 100 GB
  
• Databases hosted on a Mailbox server with continuous replication and gigabit Ethernet: 200 GB
  

  Note:
  Large databases may also require newer storage technology for increased bandwidth to accommodate repair scenarios.

  Important:
  The true maximum size for your databases should be dictated by the SLA in place at your organization. Determining the largest size database that can be backed up and restored within the period specified in your organization's SLA is how you determine the maximum size for your databases.

LCR enables log shipping on a single server. In the event of a catastrophic failure of the storage housing the active copy of the database or log files, the administrator can manually activate the passive copy of the database. The storage for the passive copy should be completely separate from the storage for the active copy. In addition:

• Controller cards should be on a different PCI bus.
  
• Each storage solution should have its own uninterruptible power supply (UPS).
  
• Each storage solution should be on a separate power circuit.
  

CCR enables log shipping to a passive node in an active/passive failover cluster. By shipping the logs to and maintaining the passive copy on a completely different server, the operational impact to the active node is decreased, and you have fault tolerance on the server.

In a geographically dispersed CCR deployment, the passive copy can be on a node that is in a different physical location from the active node, thereby providing site resiliency. Although the information in the article Deployment Guidelines for Exchange Server Multi-Site Data Replication still applies, the pull-based technology behind continuous replication means that high latency will not affect the user experience. This is a sharp contrast to the geographically dispersed cluster solution in which synchronous replication latency negatively affects the active server. With CCR, the replication process may run behind, increasing the amount of time that the active copy and passive copy are not synchronized. However, if a disaster affects the active copy, any messages that were not replicated to the passive node will be recoverable because of the transport dumpster feature of the Hub Transport server.

The hardware used for an SCC must be listed in the Cluster category of the Windows Server Catalog. The hardware used for a geographically dispersed SCC must be listed in the Geographically Dispersed Cluster category in the Windows Server Catalog to be supported.

A clustered mailbox server using shared storage has the same fundamental storage considerations as a stand-alone server. When using synchronous replication, disk latency can be artificially increased by the replication process. Care must be used to maximize the points of replication within the storage array. For more information about replication for SCCs, see Deployment Guidelines for Exchange Server Multi-Site Data Replication.