Determining the True Amount of Space in an Exchange Database
Tópico modificado em: 2006-03-08
By Jeremy Kelly
This article explains why the size of the Microsoft® Exchange database may not equal the total size of the space used by the mailboxes, and what the size listed in Event 1221 represents.
- Event 1221
- Space Management in the .stm File
- Database Space Tree Dump (Eseutil /ms)
- Why Do Exchange System Manager and Outlook Show Different Sizes for the Same Mailbox?
- For More Information
When you receive Event 1221, you see the following:
Event: 1221
Source: MSExchangeIS Public
Type: Information
Category: General
Description: The database "<storage_group>\<mailbox_store> (<server_name>)" has <nnn> megabytes of free space after online defragmentation has terminated.
Dica
Even though the event shows "terminated," this is an expected behavior.
The number of megabytes (MB) of free space presented is derived from the number of free pages that are available at the database root, messages tables, folders tables, and attachments tables. Statistically these tables represent nearly 90 percent of the space used in the database.
To understand the reason for this, a review of database space management is needed. Consider the following:
- The fundamental space allocation in Exchange is a 4 kilobyte (KB) page.
- A series of 16 consecutive pages is the amount of space that is allocated to a table when space is requested. Exchange does not allocate one page at a time.
- A page cannot store data from two different tables or belong to two different trees.
There are two levels of space management that occur in the database. One is at the global database level, which is the space available to all tables in the database store in ranges of 16 consecutive pages, and the other is at the b-tree level. If you are unfamiliar with the concepts of B+trees, it may be easier to think of them as a table. Tables maintain the state of pages they own and do not free up pages to the overall database until a 16 consecutive block of pages is freed. The reason for this is efficiency. By reusing pages that are physically adjacent to existing pages, you minimize the effort necessary to perform read and write operations. So by holding onto pages, you ensure that a larger majority of pages for a specific table are physically adjacent.
In a database, you can have thousands of tables, at least one for each folder in every mailbox. As previously mentioned, statistically the messages tables, folders tables, and attachments tables represent 90 percent of the space used in the database. These tables have the greatest percentage of free space, commonly referred to as white space, in the database.
To arrive at a calculation for the amount of available space an individual table possesses, you must interrogate the space usage information of the table. If you were to do this for thousands of tables, it would take a large amount of time to perform this calculation. As a result, Microsoft considers only the available space of the message, folder, attachment, and database root when logging Event 1221.
The database contains additional information that is not part of the mailbox allocation. For example, there are search folders, indexes, and system tables that are required for Exchange to operate. In addition, a page cannot belong to multiple tables. If a page has free space, it can only store new records for that table, and it will only contain records that meet the criteria to be stored on that page. If you stored every record beginning with the letter A on one page and every record beginning with B on a second page, there is no reason that a record beginning with A should appear on the page containing all the B values. Therefore, there can be space available on an individual page that is not taken into consideration when calculating Event 1221, because Event 1221 only adds the empty or free pages in the tables mentioned previously.
Online defragmentation compresses records to fewer pages in a particular table. For example, if a table owns 100 half-full pages, online defragmentation will attempt to free approximately half of the pages. When 16 contiguous pages are freed, the space is released to the database to be used by other tables. It is only the pages that are freed in the previously mentioned tables that eventually get reported.
Space management for the .stm file is different from the logic in the .edb file. In the .stm file, data is still arranged in 4 KB pages, however, the page state information is not maintained in the .stm file. This is done in the .edb file. Pages in the .stm file have four primary states:
- Free Currently does not contain data and is available to accept new incoming data.
- Reserved Has new data from an inbound message, but has not been committed.
- Committed Contains data currently referenced by records in the .edb file.
- Deleted A page that is being freed, but is not ready to accept new data.
Inside the .edb file, two tables exist to manage the contents of the .stm file, SLVAvail and SLVSpaceMap. In the .stm file, which is referred to as the SLV file in Extensible Storage Engine (ESE) source, space is managed in 512 page chunks. In the SLVAvail table, entries are maintained for each 512 page chunk. Two bits are used to describe the state of each page in the 512 page chunk. When a new space request is received, which is a new incoming Simple Mail Transfer Protocol (SMTP) message, this table is queried to locate space in the .stm file to store the message. A series of pages is returned to SMTP in the form of a file handle.
The SLVSpaceMap table allows for reverse lookup of pages in the database to map to a particular table, row, and column that references a particular page in the .stm file. It also records the checksum of a page in the .stm file.
If you want a more in-depth look at the space used in the database, a space dump using Exchange Server Database Utilities (Eseutil) is necessary. Note that the following example is truncated for brevity:
C:\Program Files\Exchsrvr\MDBDATA>..\bin\eseutil /ms priv1.edb
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: priv1.edb
****************************** SLV SPACE DUMP *************************
Chunk Free Res Del Com |------------ Used ------------|
=======================================================================
512 110 0 0 402 *************************
1024 0 512 0 0 ********************************
1536 512 0 0 0
2048 512 0 0 0
2560 512 0 0 0
3072 512 0 0 0
3584 512 0 0 0
4096 512 0 0 0
4608 118 0 0 394 ************************
5120 0 0 0 512 ********************************
5632 0 0 0 512 ********************************
6144 0 0 0 512 ********************************
6656 0 0 0 512 ********************************
7168 0 0 0 512 ********************************
7680 0 0 0 512 ********************************
8192 238 0 0 274 *****************
=======================================================================
TOTALS:
Free: 3538
Reserved: 512
Deleted: 0
Committed: 4142
Unknown: 0
-------------
8192
***********************************************************************
******************************** SPACE DUMP ***************************
Name Type ObjidFDP PgnoFDP PriExt Owned Available
=======================================================================
priv1.edb Db 1 1 256-m 8960 116
<SLV Avail Map> SLV 6 33 32-m 32 29
<SLV Owner Map> SLV 7 65 32-m 80 3
1-122 Tbl 75 301 8-s 8 3
MsgFolderIndex7 Idx 77 302 1-s 1 0
MsgFolderIndexPtagDel Idx 80 305 1-s 1 0
MsgFolderIndexURLComp Idx 79 304 1-s 1 0
RuleMsgFolderIndex Idx 78 303 1-s 1 0
1-23 Tbl 61 236 2-m 7778 16
<Long Values> LV 222 237 1-m 7691 8
1-24 Tbl 63 257 8-s 8 3
MsgFolderIndex7 Idx 65 258 1-s 1 0
MsgFolderIndexPtagDel Idx 68 261 1-s 1 0
MsgFolderIndexURLComp Idx 67 260 1-s 1 0
RuleMsgFolderIndex Idx 66 259 1-s 1 0
1-33 Tbl 336 8821 8-m 14 1
<Long Values> LV 342 8826 1-m 5 2
?T668f-T6654+Q3f88 Idx 354 8827 1-s 1 0
MsgFolderIndex7 Idx 338 8822 1-s 1 0
MsgFolderIndexPtagDel Idx 341 8825 1-s 1 0
MsgFolderIndexURLComp Idx 340 8824 1-s 1 0
RuleMsgFolderIndex Idx 339 8823 1-s 1 0
Folders Tbl 8 97 9-m 100 10
<Long Values> LV 105 243 1-s 1 0
*T668f+Q6749+S3001+Q6 Idx 60 232 1-s 1 0
?T668f+Q6749+S3001+Q6 Idx 59 109 1-m 9 0
Folders Fid to Pfid I Idx 17 108 1-m 9 2
FoldersIndex10 Idx 13 102 1-s 1 0
FoldersIndex13 Idx 14 103 1-s 1 0
FoldersIndex5 Idx 9 98 1-m 5 1
FoldersIndex6 Idx 10 99 1-s 1 0
FoldersIndex7 Idx 11 100 1-m 7 0
FoldersIndex8 Idx 12 101 1-s 1 0
Hashed URL Name Index Idx 15 104 1-m 5 0
ScopeFIDs DeleteTime Idx 16 105 1-s 1 0
Mailbox Tbl 21 140 2-m 10 2
MailboxIndex2 Idx 22 141 1-s 1 0
MailboxIndex3 Idx 23 160 1-s 1 0
Msg Tbl 19 112 2-m 194 63
<Long Values> LV 106 359 1-m 14 0
-----------------------------------------------------------------------
253
The first section of output is the space dump of the .stm file. The output is directly from the contents of the SLVAvail table in the .edb file. Each 2 MB-chunk, which is 512 pages, is listed to the left. The columns represent the number of pages in each state in the .stm file. The last piece of information is a graphical representation of the amount of space used in the individual 2 MB chunk.
****************************** SLV SPACE DUMP ******************************
Chunk Free Res Del Com |------------ Used ------------|
=======================================================================
512 110 0 0 402 *************************
1024 0 512 0 0 ********************************
1536 512 0 0 0
2048 512 0 0 0
2560 512 0 0 0
3072 512 0 0 0
3584 512 0 0 0
4096 512 0 0 0
4608 118 0 0 394 ************************
5120 0 0 0 512 ********************************
5632 0 0 0 512 ********************************
6144 0 0 0 512 ********************************
6656 0 0 0 512 ********************************
7168 0 0 0 512 ********************************
7680 0 0 0 512 ********************************
8192 238 0 0 274 *****************
=======================================================================
TOTALS:
Free: 3538
Reserved: 512
Deleted: 0
Committed: 4142
Unknown: 0
-------------
8192
Consider the following section of output. There are two columns, owned and available. The owned value is the number of pages in the database that contains some data. The available value represents the number of free pages available at the database root that can be distributed to tables as they need space to grow.
******************************** SPACE DUMP ***************************
Name Type ObjidFDP PgnoFDP PriExt Owned Available
=======================================================================
priv1.edb Db 1 1 256-m 8960 116
Consider the attachments table in the following section of output. There are two rows, one called 1-23, which is the primary B+tree containing actual records, and then one called <Long Values>, which contains binary blobs or fragments of records that are larger than 1 KB and cannot be stored in the primary B+tree. The value under the owned column for 1-23 is 7778. This represents the total number of pages owned by this table and all associated B+trees. This encompasses the primary B+tree, long values trees, and any other secondary indexes that could be in use by this table. The next value is the available pages, in this case 16, that can be reused by this individual table, but not by indexes or the Long Value tree. Of the 7,778 pages in use by this table, the long value tree is occupying 7,691 and there are 8 pages available:
******************************** SPACE DUMP **************************
Name Type ObjidFDP PgnoFDP PriExt Owned Available
=======================================================================
1-23 Tbl 61 236 2-m 7778 16
<Long Values> LV 222 237 1-m 7691 8
Consider a standard folder in use by someone in the database. The folder is 1-33. Note that internally, Microsoft represents tables as numbers referred to as IDs. This would typically represent a user's Inbox folder or other folder that the user created. In this case, there is a primary entry, 1-33, that is the table itself. Listed under this row are associated Long Value and indexes to this table. There are also five secondary indexes associated with this table.
Dica
Indexes come in two types, those created by schema and those created dynamically. In the following output, for example, MsgFolderIndex7 would be considered a schema index. When the Microsoft Exchange Information Store service creates this table, it also creates an index with this name. Dynamically created indexes have an unusual, but logical naming convention. In the following example ?T668f-T6654+Q3f88 is a dynamically created index with a key value being the aggregate of the columns T668f, T6654, and Q3f88.
Overall, this table occupies 14 pages with 1 page available to the primary table. The Long Value owns 5 of the 14 pages owned total by this table, and has 2 free pages, while the table entry only has one. The Owned value represents all pages in use by the table, indexes, and LV trees. The Available only represents the number of pages available to that individual index/LV/Table and is not cumulative:
******************************** SPACE DUMP ***************************
Name Type ObjidFDP PgnoFDP PriExt Owned Available
=======================================================================
1-33 Tbl 336 8821 8-m 14 1
<Long Values> LV 342 8826 1-m 5 2
?T668f-T6654+Q3f88 Idx 354 8827 1-s 1 0
MsgFolderIndex7 Idx 338 8822 1-s 1 0
MsgFolderIndexPtagDel Idx 341 8825 1-s 1 0
MsgFolderIndexURLComp Idx 340 8824 1-s 1 0
RuleMsgFolderIndex Idx 339 8823 1-s 1 0
At the end of the dump is a line similar to the following. It contains a summation of the total number of pages that are available throughout all the tables. By multiplying this value by 4,096, you determine the true amount of white space in the database:
-----------------------------------------------------------------------
253
When studying a space dump to understand where space is being used in the database, first look to the attachments table (1-23), messages table, and folders table, and determine how much space these three tables occupy. If these tables combined do not represent the majority of the size of the database, the space is being consumed by some other table or indexes. Examine the output for suspicious entries occupying a large amount of space, or many small tables combined that consume a large amount of space.
In this example, mailbox size matches the size of the database. The reason is that the tables that actually store the data could occupy a larger amount of space depending upon the density of the pages at the time, and if pages have been freed up to the database root. In most cases, this has proven to be the situation.
When a calculation is performed for a size of a mailbox, consider the following:
- Microsoft Outlook® takes into account only what a user can see from the mailbox, so only items that a user can see from the Outlook client in the mailbox folders are calculated.
- Within a mailbox, many hidden items exist. Some of these items include third-party hidden folders, rules, and items used to support the functionality included with an Exchange mailbox. These hidden items are sometimes referred to as NON_IPM_DATA. Exchange System Manager considers all related mailbox data, including NON_IPM_DATA when calculating the size of a mailbox.
This implies that the size of the mailbox is usually bigger when viewing it from Exchange System Manager, rather than viewing it from Outlook client.
There are also several bugs related to size calculations that may be a factor in understanding these discrepancies. If you incur this problem and subsequently apply a fix to correct the problem, the size counts may appear to be different until the Information Store Integrity Checker (Isinteg) is run to correct the item count issues. In general, if you are attempting to understand an item count or size issue, ensure you are running the latest version of the Microsoft Exchange Information Store service, and you have run Isinteg after applying the fix. For information about fixes that can affect mailbox size reporting, see the following Microsoft Knowledge Base articles:
- 818660, "XADM: Messaging Database Size Increases Unexpectedly After Online Maintenance and Event ID 1097 Is Logged"
- 813880, "XADM: Mailbox and Folder Sizes Are Not Updated After a Single-Instance Stored Message Is Modified"
For more information, see the following Microsoft Knowledge Base articles: