New Exchange Core Store Functionality
[This is pre-release documentation and subject to change in future releases. This topic's current status is: Editing.]

Applies to: Exchange Server 2010

Microsoft Exchange Server 2010 includes many improvements to the Exchange database architecture: 

  • Public folder reporting has been enhanced.
  • Storage groups have been removed.
  • Mailbox databases aren't connected to the server object and are now peers to the server. Database management has been moved from the Server Configuration node to the Organization Configuration node in the Exchange Management Console (EMC).
  • Reduce IOPS by 70% through investing in store schema and Extensible Storage Engine (ESE) optimizations.
    • The store schema has been flattened to remove the dependency to the server object.
    • ESE has several new improvements for high availability, performance, and database mobility.

Many of the changes to the core store functionality are high availability features. High availability has been integrated into the core architecture of Exchange 2010 to enable organizations of all sizes and in all industry segments to economically deploy a messaging continuity service.

For more information about the high availability changes in Exchange 2010, see Understanding High Availability and Site Resilience.

Enhanced Reporting for Public Folders

Public folder reporting has been enhanced to allow you to see when any changes to a public folder or to any item in the public folder were last made. You can view this information by using the Get-PublicFolderStatistics cmdlet in the Exchange Management Shell. For more information, see Exchange Management Shell.

Database Management

Databases are no longer associated with storage groups. In Exchange 2010, storage group functionality has been moved to the database.

In Exchange 2010, you can manage mailbox and public folder databases in the Organization Configuration node of the EMC. In Exchange 2007, database management was performed in the Server Configuration node.

Bb125040.note(en-us,EXCHG.149).gifImportant:
Although public folder database management has been moved from the Server Configuration node to the Organization Configuration node with the mailbox databases, the functionality of public folder databases hasn't changed in Exchange 2010. Just like in Exchange 2007, you can't create database copies of public folder databases, and you can't add public folder databases to a database availability group. Public folders still use public folder replication.
Database Cmdlet Changes

With the removal of storage groups, the storage group cmdlets were deleted and the database cmdlets now provide the functionality, as shown in the following table.

Cmdlet Change

New-StorageGroup

This cmdlet has been deleted, and configuration parameters were moved to New-MailboxDatabase and New-PublicFolderDatabase cmdlets.

Remove-StorageGroup

This cmdlet has been deleted, and configuration parameters were moved to New-MailboxDatabase and New-PublicFolderDatabase cmdlets.

Set-StorageGroup

This cmdlet has been deleted, and configuration parameters were moved to New-MailboxDatabase and New-PublicFolderDatabase cmdlets.

Get-StorageGroup

This cmdlet has been deleted, and configuration parameters were moved to New-MailboxDatabase and New-PublicFolderDatabase cmdlets.

Move-StorageGroupPath

This cmdlet has been deleted, and configuration parameters were moved to Move-DatabasePath.

New-MailboxDatabase

New-PublicFolderDatabase

These cmdlets have been extended with the parameters and functionality from New-StorageGroup. They also update the server object with a link to the new database and the database object with the hosting server name.

Remove-MailboxDatabase

Remove-PublicFolderDatabase

These cmdlets have been extended with the parameters and functionality from Remove-StorageGroup. In addition, they also update the server object with the link to the new database and the database object with the hosting server name.

Set-MailboxDatabase

Set-PublicFolderDatabase

These cmdlets have been extended with the parameters and functionality from Set-StorageGroup. When changing the host servers, they also update the server object with the link to the new database and the database object with the hosting server name.

Get-MailboxDatabase

Get-PublicFolderDatabase

These cmdlets have been extended with the parameters and functionality from Get-StorageGroup. The Status parameter is extended to return the status information currently returned by Get-StorageGroupCopyStatus.

Move-DatabasePath

This cmdlet has been extended with the parameters and functionality from Move-StorageGroupPath.

In addition to the cmdlet changes above, the StorageGroupCopy cmdlets have been deleted. For more information, see New High Availability and Site Resilience Functionality.

Store Schema Changes

The store schema has been changed to remove the dependency of mailbox databases to the server object. In addition, the new schema has been improved to help reduce IOPS by refactoring the tables used to store information and increasing the temporal locality of reference. Refactoring the tables allows higher logical contiguity and locality of reference. These changes reduce the store's reliance on the secondary indices maintained by ESE. As a result, the store is no longer sensitive to performance issues related to the secondary indices.

New Extensible Storage Engine Functionality

The Extensible Storage Engine is being improved in Exchange 2010 to achieve the following goals.

  • Larger I/O and sequential I/O to reduce IOPS
  • Optimize for commodity storage (JBOD & SATA)
  • Reduce database management
Larger and Sequential I/O

By increasing the size of the I/O and reducing the frequency of read/writes in Exchange 2010, ESE is able to increase performance. In addition, ESE is able to increase performance by making the data in the database more sequential - increasing the likelihood that related data is in the same vicinity in the B-tree.

In Exchange, all data inside the database is stored in B-trees, and the B-trees are then divided into pages. In Exchange 2010 and earlier, the data stored in the B-trees is not contiguous. In fact, previous versions of Exchange performed random read/writes to the database. This means that related data may not be in the same vicinity on the hard drive. Non-contiguous data requires more passes to read and write to the hard drive.

Space allocation

Space allocation refers to the way that ESE allocates space on the database.

Database LV Compression

A column or a record in ESE cannot span pages in the data B-tree. However, there are values in the database that break the 32 KB boundary of a page. These are referred to as long-values (LV). A table's long-value B-tree is used to store these large values.

B-Tree Defragmentation

The B-tree defragmentation process has been improved to reduce I/O operations by maintaining contiguous data in the B-tree.

B-tree defragmentation is performed in-place (as opposed to creating a new B-tree and renaming the indexes and tables) with three new operations:

  1. Page move   A page move consists of moving all data from one page to a newly allocated page.
  2. Partial left merge   A partial left merge is the same as a right merge in Exchange 2007 or earlier, except that data is moved from the left page to the right page.
  3. Full left merge   A full left is the same as a full right merge in Exchange 2007 or earlier.

Defragmentation has been changed from right merges to left merges to optimize performance. Data is read/written to the hard drive from right-to-left. If the database is being defragmented in the same direction as the read/writes, then defragmentation will conflict with the read/writes. In addition space allocation allows the next page in an extent to be allocated, but not the previous page. Because a page move needs to allocate a new page, defragging the database from left to right is much more efficient.

The Defragmentation Manager is a new event in ESE that monitors which B-trees require defragging and which B-trees have already been defragged. The defragmentation manager compiles a list of the B-trees in all mounted databases that should be defragged. As fragmented B-trees are discovered, they are registered with the Defragmentation Manager and the defragmentation manager will process them.

Page size increased to 32 KB

As stated earlier in this topic, all data inside the database is stored in B-trees, and the B-trees are divided into pages. The page size is the minimum size for reading and writing to the database; it is also the unit size used for database caching. Reading from the disk is slower than performing operations in memory; therefore, by increasing the page size to 32 KB, ESE reduces the IOPS which increases performance by caching the larger page size in memory.

Async vs. Sync DB Reads

Sync Read IO = Small/Random IO

Async Write IO = Large/Sequential IO

Exchange 2010 Store/ESE have put a lot of work in to moving away from Sync DB reads to Async DB Reads to leverage the contiguous nature of the database. View creations and updates, Outlook OST sync, Search result returns, Content Indexing crawls.

Optimize for Commodity Storage

Another of the goals of ESE in Exchange 2010 is to reduce the capital and operational costs of deploying Exchange. This can be done by reducing storage costs and optimizing for commodity storage using just a bunch of disks (JBOD) and SATA class hard drives.

Coalesced IO

Disk subsystems are more efficient at handling fewer but larger IO. As stated erlier in this topic, in Exchange 2010 or earlier the page size is the minimum read/write size and the minimum size for database caching. Coalescing IOs refers to the process for combining database page operations into a single IO operations thereby producing fewer and bigger IO operations.

Increasing the average database IO sizes has the following benefits:

  • Increased disk use efficiency   Disks are more efficient at processing large IOs. The more efficiently the disk is utilized, the more mailboxes can be hosted on that disk.
  • Increase the cache warming rate   Cache warming is a process that helps reduce the execution times by preloading the initial queries that were executed against a database the last time the database was started. After a server reboot, restart, failover, or switchover, the larger IO allows ESE to increase the rate at which the cache is warmed.
Database Maintenance

One of the goals of ESE in Exchange 2010 is to reduce the cost of maintaining and managing a database. For more information, see Mailbox Database Maintenance.

Page view tracker