Cross-Matching Exchange Databases and Log Files
Topic Last Modified: 2006-03-08
By Nino Bilic
This article covers a subject that is essential for successful disaster recovery in many situations when it is questionable if logs and databases belong together and can be used for successful data recovery.
The procedures described here need not be done every time you run recovery. In typical restore and recovery scenarios, all log files are present. But in complicated recovery scenarios in which you have manually moved log files or databases from one location to another, you can use the following procedures to ensure that you have matched logs and databases correctly.
The key to matching log files and databases is the signature. You can view log file signatures by using Exchange Server Database Utilities (Eseutil) and viewing the log file header with the command Eseutil /ml [log filename]. You can view database (.edb) and streaming database (.stm) file signatures by viewing the file header with Eseutil /mh [database filename].edb.
The following is a typical log file or database file signature:
Signature: Create time:12/17/2002 18:1:44 Rand:81060559 Computer:
When Microsoft® Exchange Server 2003 creates a new log file series starting with log file number 1, it generates a new signature. All subsequent log files in the series will have the same signature as the first log file. With this signature, you can identify immediately whether two log files belong to the same series. The signature never changes unless you delete all the log files, and start again with log file number 1.
When a new database file set is created using .edb and .stm files, Exchange again generates a new signature. This signature is not the same as the log file signature. Both the .edb and the .stm files share the same signature.
When you mount a database, thereby attaching it to a particular log stream, Exchange writes the database signature into the log file and the log file signature into the database header. You can disconnect from a database, thus detaching it from the log stream, and mount it on a different server or in a different storage group. The next time the current log fills and a new one is generated, however, the detached database will not appear in the new log file header. The database header instead reflects the signature of the log stream to which it is now attached.
|When a new series of logs is created, log Enn000001.log does not list any database signatures in its header. Enn000002.log is the first log to list databases in the header.|
When a database is attached to an existing log file, Exchange does not write the database signature to the log file header, but lists it in an attach record in the middle of the log file. Exchange generates a log file header at the moment a log file is created, and never updates it after that. A newly attached database appears in the header of the next log file because Exchange records in the log header only signatures of databases attached at the moment a new log is created.
A single stream of log files may have multiple databases attached to it at various times. The log signature stays constant regardless of how many databases are attached to the series of logs. Typically, Exchange interleaves operations bound for multiple databases within a given transaction log file. During log file replay, each database applies only the transactions applicable to itself.
Unlike a log signature, database signatures can change over the lifetime of the database. A database signature changes if the database is repaired using Eseutil /p or if an offline defragmentation using Eseutil /d is performed. The signature is deliberately changed after these operations to prevent replay of log files associated with the old database signature. With log file replay, repairing or defragmenting a database with Eseutil.exe results in a different database than existed before the operation.
Suppose that you have a database and a set of log files. To find out if these logs are associated with this database, do the following:
Examine the database header with Eseutil /mh, paying particular attention to these three lines:
- DB Signature
- Log Signature
- Log Required
Use the information you found in step 1 to determine which log file is most likely the first one needed for replay. If you cannot find a matching numbered log file, check the lGeneration value of the Enn.log file to discover if it is a match.
Match the signature of this log file against the Log Signature value from the database header. If it is a match, it is likely that this is the log file needed. This is not a completely reliable test, but it is almost always valid. If an administrator were to mount multiple copies of log files and databases against each other, and then mix the file sets in the right way, a log file that was not a match but that passed the tests could be generated.
Verify that subsequent available logs all belong to the same series. Use the command Eseutil /ml Enn to scan the current folder for all log files whose names begin with Enn and report damaged logs, creation time or Prev Gen Time mismatches, and missing log files.
Another way to do this is to examine each log header and verify that all signatures match in all logs, that the lGeneration values inside the log headers are in unbroken sequence, and that the Creation Time value in each header matches the Prev Gen Time for each subsequent log.
This section contains examples of how you can use Eseutil.exe to match databases and log files as described earlier.
Use Eseutil /mh to examine the database header. You are looking for three particular lines:
- DB Signature
- Log Signature
- Log Required
DB Signature: Create time:12/30/2002 16:51:49 Rand:606486607 Computer:
Log Signature: Create time:12/30/2002 17:01:27 Rand:607052340 Computer:
Log Required: 44-46
If a database has been disconnected normally, the Log Required value is 0-0, meaning that no soft recovery is required to start the database. In this example, the database was unexpectedly stopped, and logs 44, 45, and 46 are required for recovery.
The Log Required range is in decimal format, so you must convert the numbers to hexadecimal format to correlate them with actual log file names. In this case, the hexadecimal equivalents of 44, 45, and 46 are 2c, 2d, and 2e. Exchange therefore typically names the required logs Enn00002c.log, Enn00002d.log, and Enn.log. When Enn.log is full, Exchange automatically renames it Enn00002e.log. You can verify this by examining the lGeneration line of the Enn.log header, which reads:
lGeneration: 46 (0x2E)
You know from the Log Required values shown that you must have at least these three log files to restart this database. If you use Eseutil /mk and read the header of the checkpoint file, you will see that the checkpoint is at log 44, which is 0x2c. The Log Required value acts as an internal checkpoint in the database file itself, listing not only the first log required for recovery, but also the last log.
If the Log Required line reads 0-0, you do not need any log files to restart the database. Nonetheless, you can play log files forward into the database, as long as you have the Enn.log to which the database was attached when last shut down. If you have this log file and subsequent log files, you can play forward as long as log files are available. To do this, there must be no checkpoint file, or the checkpoint must be at or before the lowest log required.
|There is no internal value in the database header that tells you which log files are required for replay when the database has been shut down cleanly, because no log files are required. However, you may still want to play additional log files. The Last Consistent value in the database header lists the log file that was in use at the time the database was last running. You can start with this log file and play forward. In Exchange Server 5.5, if you made a guess about which log file to start with, you could damage the database by attempting replay. Starting with Exchange 2000 Server, additional safeguards prevent inappropriate log file replay without damaging the database.|
Eseutil.exe can detect mismatched or missing logs in a sequence without examining individual headers. The command Eseutil /ml Enn scans the current folder for all log files whose names begin with Enn and reports damaged logs, creation time or Prev Gen Time mismatches, and missing log files. The following example shows typical output of this command, with each kind of error shown:
D:\exchsrvr\MDBDATA\log3\test>eseutil /ml e00
Microsoft(R) Exchange Server Database Utilities
Copyright (C) Microsoft Corporation 1991-2000. All Rights Reserved.
Initiating FILE DUMP mode...
Verifying log files...
Base name: e00
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000001.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000002.log
ERROR: Log damaged (unusable). Last Lgpos: (0x2,1059,0). Error -501.
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000003.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000004.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000005.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000006.log
ERROR: Invalid log sequence. Previous generation time is [12/27/2002 13:15:24], but expected [12/30/2002 17:03:33].
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000007.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000008.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000009.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E000000A.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E000000B.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E000000C.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E000000D.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E000000E.log - OK
Missing log file: e000000f.log
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000010.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E0000011.log - OK
Log file: D:\exchsrvr\MDBDATA\log3\test\E00.log - OK
Operation terminated with error -501 (JET_errLogFileCorrupt, Log file is corrupt) after 22.272 seconds.
If you are scanning a large number of log files, it may be helpful to redirect the output of the command to a text file for review. For example:
ESEUTIL /ML E00 > LOGCHECK.TXT
For more information, see Using Exchange Server 2003 Recovery Storage Groups.