White Paper: Exchange 2007 Transport Journaling

White Paper: Exchange 2007 Transpor...

Collapse All

Topic Last Modified: 2008-10-27

Bb738122.d7fd032c-98cc-4c22-b73e-ad0211c10857(en-us,EXCHG.80).jpg Naveen Chand - Program Manager, Microsoft Exchange Server

September 2007

Summary

Microsoft Exchange Server 2007 includes the transport journaling feature. Journaling is a tool that many organizations use to help meet their compliance requirements for capturing e-mail communications. This white paper supplements the Exchange 2007 Help documentation by providing additional design details and recommended deployment practices for transport journaling. For more information, see the Exchange Server 2007 Help.

Note:
To print this white paper, click Printer Friendly Version in your Web browser.

Applies to:

Release to manufacturing (RTM) version of Microsoft Exchange Server 2007 and Exchange Server 2007 Service Pack 1 (SP1)

Table of Contents

Introduction
How Exchange 2007 Journaling Works
Things to Consider When You Create Journal Rules
Multiple Journal Reports
Journaling in Various Topologies
Journaling Deployments
Journal Reports
Journal Report Format
Error Handling
Performance Monitoring

Introduction

Information retention is one of the primary focuses of regulatory compliance. Laws such as the Sarbanes-Oxley Act of 2002 (SOX) and Security Exchange Commission Rule 17a-4 (SEC Rule 17 A-4) establish retention requirements for e-mail communications. The transport journaling feature of Exchange 2007 makes it easy to create and administer policy to help achieve compliance to information retention regulations. Administrators can create journal rules for specific users or groups. These rules examine each message as the message flows through the Exchange organization and determine whether the message should be journaled.

Note:
Granular journal rules are available with premium journaling only. To use premium journaling, you must have the Exchange Server Enterprise Client Access License (CAL).

For messages that must be retained, Exchange 2007 creates a journal report. Journal reports contain verbose information about the sender and recipient. This information can later be used for e-mail forensics. Journal reports also contain an attached copy of the original message.

How to Capture Messages for Compliance Purposes in Exchange 2007

The Exchange 2007 journaling feature was designed specifically to enable organizations to capture the most complete set of information with the highest possible fidelity. Because of the huge risks involved with losing journal reports, safeguards have been built into Exchange 2007 journaling to help make sure journal messages are not lost, even if there is a configuration problem. In addition, if the Journaling agent is enabled and it encounters a configuration problem, the Journaling agent forces messages to be queued on the Hub Transport server until the problem is resolved. This ensures that messages do not pass through the system without the Journaling agent processing them.

How Exchange 2007 Journaling Differs from Exchange 2003 Journaling

Microsoft Exchange Server 2003 provided three types of journaling functionality that could be enabled.

Message-only journaling This functionality was introduced in Microsoft Exchange Server version 5.5 Service Pack 1. In Exchange 2000 Server and later versions of Exchange Server, message-level journaling creates a copy of all messages and the message sender and message recipient data from the message header for messages that are sent from mailboxes or sent to mailboxes on a given mailbox database and sends the message copies to a specified mailbox. The message header only contains the message recipient data that the sender declared to the recipients when the message was created. If an external message is received from the Internet, the message envelope sender and message envelope recipients are journaled. The message envelope is the address information that is used by Simple Mail Transfer Protocol (SMTP) messaging servers to route mail. By default, when message-only journaling is enabled, Exchange 2003 does not account for blind carbon-copy (Bcc) recipients, recipients from transport forwarding rules, or recipients from distribution group expansion.
Bcc journaling Bcc journaling is like message-only journaling with the added ability to capture the Bcc recipients. When Bcc journaling is enabled, Exchange Server captures all the recipients that are known at the originating server. This includes Bcc recipients. If the recipient list includes hidden distribution groups, query-based distribution groups, or distribution groups that are expanded on another server, the recipients for these distribution groups are not included in the journaled mail. This functionality is enabled by setting a registry key. For more information, see Microsoft Knowledge Base article 810999, Bcc information is lost for journaled messages in Exchange 2000.
Envelope journaling Envelope journaling differs from message-only journaling and Bcc journaling because it lets you archive transport envelope information, also known as message headers. This includes information about the recipients who actually received the message, including Bcc recipients and recipients from distribution groups. Envelope journaling delivers messages that are flagged to be archived by using an envelope message that contains a journal report together with the original message. The original message is delivered as an attachment. The body of the journal report contains the transport envelope data of the archived message.

Although these three journaling variations exist in Exchange 2003, most regulations that require journaling rely on envelope journaling for compliance. This is one of the primary reasons that Exchange 2007 supports only envelope journaling.

Although at first glance Exchange 2007 journaling may seem similar to Exchange 2003, the Exchange 2007 implementation is very different from its predecessor. In Exchange 2007, journaling has moved off the mailbox and now runs completely in transport on the Hub Transport server role. This change has enabled a variety of design improvements, such as making sure that a non-delivery report (NDR) is never sent for a journal report, sending journal reports directly to any SMTP address, and more precise control over which users are journaled. Unlike Exchange 2003, in which journaling was only configurable on a per-mailbox database level, Exchange 2007 journaling can be enabled for individual recipients, members of a group, the whole organization, and on a per- mailbox database basis. In addition, in Exchange 2007, the journal report format has also been modified to include detailed To, Cc, Bcc, Alt-Recipient, and distribution group expansion information. For more information, see Understanding Journal Reports.

Return to top

How Exchange 2007 Journaling Works

Let's use the following scenario to understand how journaling works in Exchange 2007: The Chief Executive Officer (CEO) of a multinational company sends e-mail to a distribution group named JPN-DL@contoso.com. The JPN-DL distribution group includes all employees in the Japan division, which is an Exchange site. The JPN-DL distribution group has a designated expansion server that is located in the Japan site called JPN-EXPN.

The following configuration exists:

The administrators of the company have configured a rule to journal all e-mail to and from the CEO to an offsite journaling archive that is run by an offsite journaling storage vendor via SMTP.
All users of JPN-DL are located on the JPN-Mbx server.
JPN-HUB server is the Hub Transport server for the JPN site.

The following figure illustrates this scenario.

Journaling on Exchange 2007
Exchange 2007 journaling scernario

In this scenario, the following events occur when the CEO sends a message that is addressed to a user who is a member of the JPN-DL distribution group:

The CEO’s Hub Transport server receives the message, and the transport agents all run.
The Journaling agent fires, determines that the message must be journaled, and creates a journal report for the message that is addressed to the offsite journaling storage vendor.
The journal report is delivered to the offsite journaling archive.
The original message is also sent to the JPN Hub Transport server.
The JPN Hub Transport server receives the original message, and all the transport agents are run. The server determines that this message has already been journaled so that it does not have to journal it again.
The original message is delivered to the JPN-MBX server.

The Journaling Agent in Exchange 2007

In Exchange 2007, the transport journaling feature has been implemented as a Transport agent. For more information about transport agents, see Overview of Transport Agents. For more information about how these agents fit into the overall processing of messages in Exchange 2007, see Microsoft Exchange Server 2007 Transport Architecture Diagrams.

By default, the Journaling agent is installed on each Hub Transport server and is registered on both the OnSubmittedMessage and OnRoutedMessage events in the categorization part of the transport pipeline.

When the Journaling agent first loads, it compiles all the journal rules. The journaling process occurs in the following two stages:

The Journaling agent decides whether the journal rules should be run. The Journaling agent first checks whether the mailbox databases of the sender or any of the recipients are configured for journaling. If not, the Journaling agent evaluates the journal rules. The Journaling agent concatenates all the journal rules and evaluates the whole set of conditions at the same time. If any of the conditions evaluate to true, the message must be journaled. Executing the journal rules causes the journal-target-recipients that are specified in the journal rules to be added in the extended property Microsoft.Exchange.JournalRecipientList on the transport mail item as a list of mailboxes.
For Exchange 2003 compatibility, the Microsoft.Exchange.JournalRecipientList property is also added as a transmittable XEXCH50 property on the transport mail item. Although Exchange 2007 ignores this property, Exchange 2003 requires it to make sure duplicate reports are not generated.
A journal report is generated and sent to all the e-mail addresses that are collected in the journal-target-recipient list.

Rule Configuration

Journal rules are stored in the Active Directory directory service under the CN=Journaling, CN=Rules, CN=Transport Settings, CN=First Organization, CN=Microsoft Exchange, CN=Services, CN=Configuration, DC=… container.

When you create a new rule by using the New Journal Rule wizard in the Exchange Management Console or by using the Exchange Management Shell, the rule is published to Active Directory and is eventually propagated to each Hub Transport server. The time before each Hub Transport server picks up the Active Directory change depends on Active Directory latency. Active Directory latency varies depending on the topology configuration.

Because the journal rules configuration may change after the Journaling agent has started, the Journaling agent registers for Active Directory change notification events. After a new configuration is picked up by a Hub Transport server, the Journaling agent reloads the set of journal rules.

Important:
As noted earlier, the exact time that the Journaling agent takes to pick up a configuration change varies depending on the Active Directory replication latency in the organization. The Active Directory replication latency in the organization varies depending on the topology configuration.

When Journal Rules Are Run

In Exchange 2007, journaling has been designed to journal messages as early as possible because messages may travel through multiple Hub Transport servers before they reach their final destination.

When a message enters the system, either through local submission, (submitted by a user in the organization) or through a Hub Transport server from an Internet gateway, the Journaling agent inspects the message for the X-Microsoft-Exchange-ProcessedByJournaling header.

If the X-Microsoft-Exchange-ProcessedByJournaling header does not exist, the Journaling agent recognizes that the message is on its first hop. The Journaling agent then runs the journal rules and stamps the X-Microsoft-Exchange-ProcessedByJournaling header on the message.

Note:
The first Hub Transport server that is encountered is considered the first hop, even if the message comes from an Exchange 2003 server.

If the X-Microsoft-Exchange-ProcessedByJournaling header already exists on the message, the Journal rules are run again only if there is a change to the set of recipients on the message (that is, the message header recipient list). Such changes can occur in the case of distribution group expansion, a transport rule, third-party agents, and so on.

Detecting changes by the Categorizer or Routing

The Journaling agent is registered on both the OnSubmittedMessage and OnRoutedMessage events to make sure that any changes to the set of recipients by the categorizer or routing during this time are detected. The Journaling agent collects the recipient SMTP addresses during the OnSubmittedMessage and OnRoutedMessage events and runs a comparison to determine whether any recipients were added or changed. If any changes have occurred, the journal rules must be rerun.

Return to top

Things to Consider When You Create Journal Rules

As previously mentioned, Exchange 2007 lets administrators create precise rules to either journal messages that are sent to or received from an individual recipient, such as a mailbox or contact object, any member of a distribution group, such as all members of the Executives distribution group, everyone on a particular mailbox database, or everyone in the organization. Although many of these rules are configured in a similar manner, the performance implications of each are very different. For more information, see Managing Journal Rules.

In this section, I describe some important scenarios and the best practices that we recommend.

Journaling for an Individual Recipient

When a journal rule is designed for an individual recipient, the destination address for sending journal reports must also be specified. Although this kind of rule provides the flexibility of specifying a unique “Send Journal Report To” address, creating lots of rules can become cumbersome to manage and also generate more overhead on the system as each rule condition must be processed.

Best Practice

For organizations in which lots of users must be journaled, the recommended approaches are either of the following:

Enable journaling for the whole organization.
Design a journal rule that uses distribution groups.

We do not recommend that you use one journal rule per user.

Journaling for Members of a Distribution Group

When a journal rule is designed for an individual recipient, you must specify the destination address for sending the journal reports. To reduce the number of Active Directory queries, the distribution group membership is cached in memory for four hours. Therefore, any changes to the distribution group, such as the removal or addition of members, is not picked up until either the cache is refreshed in four hours or the Microsoft Exchange Transport service is restarted. Organizations should consider this when they decide which journal rules to create.

Best Practice

If distribution groups in the organization frequently change but require strict journaling, we recommend that you not use membership in distribution groups as the criterion for journaling. Instead, we recommend that you take advantage of store journaling, for which the cache is updated every 10 minutes, or organization-wide journaling.

Using a Single Journal Rule for the Whole Organization

When you create a journal rule by using the Exchange Management Console, if you do not select the Journal messages for recipient check box, the rule will apply to all messages in the organization. If you don't specify the Recipient parameter when you use the New-JournalRule cmdlet in the Exchange Management Shell, the rule will apply to all messages in the organization. Therefore all messages will be journaled. Although no data is cached for organization-wide journal rules, each rule can only specify a single destination address for journal reports.

Best Practice

Because organization-wide journal rules apply to all messages, you should accurately account for the volume of messages that will be sent to the journaling mailbox or the third-party e-mail recipient that is designated in the Send Journal reports to e-mail address field.

Mailbox Database-Based Journaling

As in Exchange 2003, in Exchange 2007, you can also configure journaling on the mailbox database level. In Exchange 2007, although the mailbox database-based journaling configuration is set on the mailbox database directly, journaling is performed by the Journaling agent during categorization.

Best Practice

Use the mailbox database-based journaling option for organizations in which the recipients that are journaled frequently change. The Journaling agent caches the journaling state of the mailbox database in memory for 10 minutes. Therefore, after you modify the journaling configuration of the mailbox database, it can take up to 10 minutes for journaling to go into effect. In addition, Active directory replication latency is also a factor that you must consider.

Return to top

Multiple Journal Reports

As mentioned earlier in this white paper, journaling in Exchange 2007 is implemented as a transport agent during categorization. The Journaling agent examines every message as it flows through the transport pipeline. Although journaling in Exchange 2007 never creates duplicate journal reports, if a message is bifurcated, or split, before the Journaling agent examines it, multiple journal reports are generated. This is because, to the Journaling agent, each bifurcated copy of the message is a “new message” that must be evaluated.

The categorizer may bifurcate a message before the Journaling agent examines it for several reasons, as in the following examples that apply to the release to manufacture version of Exchange 2007:

The chipping size of the distribution group For performance reasons, the categorizer breaks messages that have more than 1,000 recipients into multiple messages, each of which contains 1,000 users or less.

Note:
The value 1000 is the default value for the categorizer chipping size. This value is controlled by the ExpansionSizeLimit parameter in the EdgeTransport.exe.config application configuration file. We recommend that you do not modify this value on an Exchange 2007 transport server in a production environment.

For example, if a message is sent to a distribution group that has 1,300 recipients, and the sender is enabled for journaling, two journal reports are generated:

One has 1,000 recipients.
Another has the remaining 300 recipients.

Both a distribution group and individual recipients are addressed on the submitted message In order for transport to send additional distribution group-specific properties, such as suppress DSNs, messages that contain both a distribution group and individual recipients are split into multiple messages, one that contains the distribution group and the other that contains the individual recipients.
Two distribution groups are addressed on the submitted message As in the previous case, transport may send distribution group-specific properties for each distribution group. Therefore, the categorizer bifurcates the original message so that each distribution group is separated into a new message.
Alternative recipients if an alternative recipient is present, the categorizer bifurcates the message and includes the alternative recipient on the second message.
Content conversion If a content conversion must occur for some recipients, for example, when a message is sent to both internal and external recipients, multiple journal reports are generated. Multiple journal reports are also generated if a message is sent to a distribution group for which there is a different distribution group expansion server. Because journaling fires on the first hop, the message is journaled. Then when the message is relayed to the distribution group expansion server, a second copy of the message is journaled after the distribution group recipients have been expanded. For more information, see Understanding Content Conversion.

In all these examples, multiple journal reports are generated. Even though each report contains the same attached copy of the original message, the report bodies differ because each report body contains a different set of message header recipients. To capture a complete set of information, the information that is contained in all journal report bodies must be captured.

Exchange 2007 Service Pack 1 Update

In Exchange 2007 SP1, the Journaling agent has been updated so that multiple journal reports are generated only in the following circumstances:

The chipping size of the distribution group warrants multiple journal reports.
A message includes a distribution group among the addressees and is delivered to a Hub Transport server that is not the expansion server for that distribution group. In this case, the Hub Transport server sends the message to the Hub Transport server that is responsible for the expansion. That Hub Transport server expands the distribution group, the recipients list changes, and journaling is run again on that message.

Return to top

Journaling in Various Topologies

In this section, I explain how Exchange 2007 transport journaling functions in various deployments.

Exchange 2007 to Exchange 2007: Cross-Site Topology

A message may go from one Exchange 2007 Hub Transport server to another Hub Transport server because the sender and recipients are in different Active Directory sites. For example, if the sender is enabled for journaling with an Exchange 2007 journal rule, the Journaling agent is invoked for the message on both Exchange 2007 hops. However, the Journaling agent keeps track of the following conditions:

Has this message already been on an Exchange 2007 hop before? In this scenario, this means that the X-ProcessedByJournaling header already exists on the message.
Have the message recipients changed? In this scenario, the recipients have not changed.

The Journaling agent determines that the journal rules were already run on the first Exchange 2007 hop and you do not have to run the journal rules again. Therefore, only the first hop generates a journal report.

Exchange 2007 to Exchange 2007: Distribution Group Expansion Server

If the sender and recipients are enabled for journaling in the first hop, that hop generates a journal report. When the message is forwarded to the second hop for distribution group expansion, the Journaling agent checks the following conditions again:

Has this message already been on an Exchange 2007 hop before? In this scenario, it has.
Has this message already been on an Exchange 2007 hop before? In this scenario, it has been on an Exchange 2007 hop before.
Have the message recipients changed? In this scenario, the recipients have changed.

Therefore, even though journal rules have already run on this message, the message itself has changed because the set of recipients has changed. A new recipient that is also configured to be journaled may have been added. The second hop has to report that the new recipient was sent this message and therefore the journal rules are reapplied on the second hop.

Exchange 2003 to Exchange 2007: Sender Journaled

A user on Exchange 2003 is enabled for journaling by configuration of the Exchange 2003 mailbox database to send journal reports to e2k3journal@fabrikam.com. This user sends a message to an Exchange 2007 recipient. The Exchange 2007 recipient is also enabled for journaling, only on Exchange 2007, and has a journal rule created to send journal reports to e2k7journal@fabrikam.com.

How does this work?

Exchange 2003 journals this user to e2k3journal@fabrikam.com and puts the GUID of the journal recipient in the XExch50 property.
The Exchange 2003-generated journal report is delivered to e2K3journal@contoso.com.
Exchange 2007 encounters the message and determines that this is the first-hop Hub Transport server, that is, the first Exchange 2007 server to be encountered.
1. The Journaling agent runs the legacy journaling code that checks the Exchange 2003 mailbox database configuration of the sender and adds e2k3journal@fabrikam.com to the list of journal recipients.
2. The Journaling agent runs the journal rule on all recipients and adds e2k7journal@fabrikam.com to the journal recipient list.
3. The recipients sent by Exchange 2003 in the XExch50 property are subtracted from this set, leaving only e2k7journal@fabrikam.com. Therefore, e2k3journal@fabrikam.com does not receive duplicate journal reports.
The Exchange 2007 generated journal report is delivered to e2k7journal@fabrikam.com.

The following diagram illustrates this scenario and the associated interoperability logic.

Exchange 2003 to Exchange 2007: Sender journaled

Exchange 2007 to Exchange 2003: Recipient Journaled

This scenario reverses the previous scenario in which the sender is journaled. In this scenario, the Exchange 2003 recipient is enabled for journaling and an Exchange 2007 sender sends to the Exchange 2003 user.

Exchange 2007 generates a journal report for the Exchange 2003 recipient by reading the Exchange 2003 journaling configuration.
Exchange 2007 sets the XExch50:JournalRecipientList property so that Exchange 2003 knows that Exchange 2007 has already sent a journal report for the Exchange 2003 recipient.
Exchange 2003 runs journaling again and determines that the Exchange 2003 recipient should be journaled. However, it sees that the journal recipient is already in the XExch50 property, and no report is generated.

Exchange 2007 to Exchange 2003: Both Sender and Recipient Journaled

Exchange 2007 generates a report for both the sender and recipient. The XExch50:JournalRecipientList property is set by Exchange 2007. It indicates to Exchange 2003 that this message has already been processed for journaling on a previous hop. Exchange 2003 will not try to journal the message again unless it adds more recipients.

Return to top

Journaling Deployments

Because journal reports contain confidential information, they must be transmitted securely. The following sections describe the recommended steps for configuring journaling to an external repository in various scenarios.

Journaling to a Trusted Exchange Mailbox

In deployments where mail is journaled to a trusted Exchange mailbox in the Exchange organization, the journal mailbox should be locked down so that only journal reports that are generated by the Exchange Server can be sent to the archive. This method ensures that journal reports are not spoofed. The following procedure describes the high-level steps that are required to lock down the journal mailbox:

Create a journal mailbox on the Exchange server that you are using for journaling.
To make sure that journal reports are not spoofed, follow these steps to set permissions on the journal mailbox so that only Exchange Server can send to it.
1. In the console tree, expand Recipient Configuration.
2. In the result pane, select the Journal Mailbox.
3. In the action pane, under Journal Mailbox, click Properties, and then select the Mail Flow Settings tab.
4. Select Message Delivery Restrictions from the list of mail flow settings.
5. Add Microsoft Exchange to the senders.
  
  Note:
  The Microsoft Exchange recipient is only visible after Exchange 2007 SP1 is installed. In Exchange 2007 RTM, select the Administrator account.
6. Select the Require that all senders are authenticated check box.
Hide the journal mailbox from the GAL by using the General tab on the Properties page.
Set the journal mailbox as the destination in the journal rule.

Note:
The Microsoft Exchange recipient is only visible after Exchange 2007 SP1 is installed. In Exchange 2007 RTM, select the Administrator account.

The following figure shows this procedure.

Journaling to a trusted Exchange mailbox
Setting Authentication for a Journal Mailbox

Journaling to an External Organization

In deployments where journal reports are sent to an external archive, such as an off-site archive vendor, organizations should ensure the following conditions are true:

Journal reports sent to the external archive are transmitted over an encrypted channel.
The off-site repository is locked down so that only journal reports from partners are accepted. For example, the off-site archive vendor called MyHostedArchive.com locks down their store for Contoso.com so that only journal reports that are generated from Contoso.com are stored there.
The off-site archive is locked-down so that employees in an organization cannot spoof journal reports. For example, the off-site archive vendor MyHostedArchive.com locks down their store for Contoso.com so that it only accepts journal reports that are generated by Exchange servers that belong to Contoso.com. This ensures that employees in Contoso.com cannot send spoofed journal report messages to the archive.

The following procedure describes one method to help secure journaling to an external repository by using Exchange 2007.

Note:
It is possible to help secure delivery of journal reports to an external archive by using methods other than this procedure. To help secure journaling content, organizations should work with their archiving vendors to ensure that the three conditions described earlier in this section exist.

Configure routing in Contoso.com to send journal reports over a Send connector on which Transport Layer Security (TLS) is enabled, such as only for the MyHostedArchive.com address space.
Configure MyHostedArchive.com to accept messages from Contoso.com over a Receive connector on which TLS is enabled.
Create a contact in Contoso.com that points to the journal mailbox in MyHostedArchive.com.

To make sure that journal reports are not spoofed, follow these steps to set permissions on the contact so that only Exchange Server can send to it:

In the console tree, expand Recipient Configuration.
In the result pane, select the Journal Mailbox.
In the action pane, under Journal Mailbox, click Properties, and then select the Mail Flow Settings tab.
Select Message Delivery Restrictions from the list of mail flow settings.

Add Microsoft Exchange to the senders.

Note:
As noted earlier in this white paper, the Microsoft Exchange recipient is only visible after Exchange 2007 SP1 is installed. In Exchange 2007 RTM, select the Administrator account.

Select the Require that all senders are authenticated check box.

Hide the journal mailbox from the GAL by using the General tab on the Properties page.
Configure domain-secured e-mail between Contoso.com and MyHostedArchive.com.
Create a contact in MyHostedArchive.com for Contoso.com.
- Set the contact’s address to MicrosoftExchange@Contoso.com.
Set permissions on the journal mailbox so that only the contact created in the previous can send to it.

The following figure shows this procedure.

Journaling to an external organization
Configuring secure journaling to an external store

Full Trust between Two Organizations

In deployments where journal reports are sent between two organizations that have full trust between them (so you do not have to encrypt messages), you can follow these steps:

Create a contact in Contoso.com that points to the journal mailbox in Contoso.Archive.com.

To make sure that journal reports are not spoofed, follow these steps to set permissions on the contact so that only Exchange Server can send to it:

In the console tree, expand Recipient Configuration.
In the result pane, select the Journal Mailbox.
In the action pane, under Journal Mailbox, click Properties, and then select the Mail Flow Settings tab.
Select Message Delivery Restrictions from the list of mail flow settings.

Add Microsoft Exchange to the senders.

Note:
As noted earlier in this white paper, the Microsoft Exchange recipient is only visible after Exchange 2007 SP1 is installed. In Exchange 2007 RTM, select the Administrator account.

Select the Require that all senders are authenticated check box.

Configure header firewall in Contoso.com to allow all organization headers to flow between Contoso.com and Contoso.Archive.Com
Create a contact in Contoso.Archive.com for Contoso.com.
- Set the contact’s address to MicrosoftExchange@Contoso.com.
Set permissions on the journal mailbox so that only the contact created in the previous step can send to it.

The following figure shows this procedure.

Full trust between two organizations
Configuring full trust journaling

Does Journaling Capture Everything?

Exchange 2007 transport journaling captures all messages that flow through the Exchange 2007 Hub Transport servers except for the following message types:

Public folder replication messages
Journal reports
NDRs of journal reports
Quarantined messages

By default, journaling does not also process voice mail messages and missed call notification messages that are handled by an Exchange 2007 Unified Messaging (UM) server. However, you can configure journaling of these messages by setting the VoicemailJournalingEnabled parameter to $true by using the Set-TransportConfig cmdlet in the Exchange Management Shell. For more information, see How to Enable or Disable Journaling of Voice Mail and Missed Call Notifications.

Return to top

Journal Reports

New Recipient Origination History in Exchange 2007

In Exchange 2003, the envelope recipients of a message are listed in the journal report without any additional data. This can be a problem when a recipient is Bcc’ed or added to the envelope because of distribution group expansion. With Exchange 2003 envelope journaling, it is impossible to tell by just looking at the journal report whether an envelope recipient that is not a message header recipient on the original message is a recipient from an expanded distribution group or a Bcc recipient.

Exchange 2007 extends the journal report format to include this information. For example, the following lines in the journal report may indicate that bccrecip@contoso.com was a Bcc recipient, and dlmember@contoso.com was added to the message because of the expansion of a distribution group named dlname@contoso.com:

Bcc: bccrecip@contoso.com

To: dlmember@contoso.com, Expanded: dlname@contoso.com

With Exchange 2007 journaling, the goal is to verify that bccrecip@contoso.com and dlmember@contoso.com match the recipient addresses that appear in the message header of the message that was journaled. Therefore, these extra headers let you correlate between the journal report and the original message.

Improved Fidelity of Journal Reports in Exchange 2007

A common deployment practice for journaling with Exchange 2003 is to direct journal reports to a local mailbox server. Then the messages are rerouted to a long-term off-site archive. When a message moves from a mailbox to transport, it undergoes a translation from MAPI to Summary Transport Neutral Encapsulation Format (S/TNEF). By default, Exchange is configured to convert S/TNEF to MIME when delivering externally. Conversion from S/TNEF to MIME is a lossy conversion. Therefore, when journal reports are delivered to an external archive and no configuration changes are made to the connectors, the journal reports may contain less information.

In Exchange 2007, journal reports are fully formed in transport. Therefore, there is no need to deliver to an Exchange mailbox first. Also, the envelope attachments of Exchange 2007 journal reports are captured in S/TNEF. Because S/TNEF is a richer format than MIME, journal reports captured in S/TNEF maintain better fidelity of the original message. For example, information, such as voting buttons, is not lost.

Note:
Although many archiving vendors are starting to support the improved fidelity journal attachment format, S/TNEF, Exchange 2007 SP1 includes functionality that lets you convert journal reports from TNEF to MIME, much like in Exchange 2003. This functionality was reintroduced to provide organizations with an easier migration path for their archiving solutions. With Exchange 2007 SP1, you can enable delivery of S/TNEF journal reports by modifying settings on the Send connector.

Note:

Although many archiving vendors are starting to support the improved fidelity journal attachment format, S/TNEF, Exchange 2007 SP1 includes functionality that lets you convert journal reports from TNEF to MIME, much like in Exchange 2003. This functionality was reintroduced to provide organizations with an easier migration path for their archiving solutions. With Exchange 2007 SP1, you can enable delivery of S/TNEF journal reports by modifying settings on the Send connector.

Return to top

Journal Report Format

In Exchange 2007, the journal report format is similar to but incompatible with Exchange 2003. The format of the journal report is specified as follows:

The Journaling agent writes the body of the report according to the Augmented Backus-Naur Form (ABNF) syntax that is specified later in this section.
The Journaling agent automatically picks the most appropriate character set based on the subject character set and the character set of the original message.
The journal report is identified by the presence of an empty header X-MS-Exchange-Organization-Journal-Report, which is converted to X-MS-Journal-Report when sent outside the organization or viewed from an end user's mailbox. This header identifies the message as a journal report and enables the system to treat it specially, for example, bypassing restrictions on size. This header is firewalled by header firewall to prevent it from appearing in unauthorized messages. Only the system (the Journaling agent) can set this header. Header firewall enforces it. For more information about header firewall, see Understanding Header Firewall.
For backward compatibility with Exchange 2003, the PR_CONTENT_IDENTIFIER XEXCH50 property is set to the string ExJournalData. It does this in the transport pipeline. The PR_CONTENT_IDENTIFIER XEXCH50 property informs Exchange 2003 that a message is a journal report.
The RFC 2822 To:, From:, and Subject: headers of the journal report message are set identically to the respective headers of the message that is being journaled. This is backward compatible with Exchange 2003 and enables easy search by using Outlook. In addition, the Sender: header is set to Microsoft Exchange.
The original message is attached as an S/TNEF attachment, or a MIME attachment if the message originates from the Internet, to the journal report.

ABNF Specification

The following is the ABNF specification.

Journal-report-body ::==

"Sender:" SP <smtp-address> CRLF

"Subject:" SP <subject-string> CRLF

"Message-ID:" SP <message-ID> CRLF

[<recipient-specification>]+

<subject-string> may contain non-ASCII characters. It may contain characters from the character set that is specified for the journal report.

<smtp-address> is an SMTP e-mail address as defined in RFC 2822.

<message-ID> is the SMTP message-ID of the message that is being journaled.

One or more occurrences of <recipient-specification> give information about the envelope recipients. The following provides details about <recipient-specification>:

<recipient-specification> ::==

<recipient-p2-type> ":" SP <recipient-address>

[<redirection-type> ":" SP <source-address>] CRLF

<recipient-p2-type> ::== "Bcc" | "To" | "Cc" | "Recipient"

<recipient-address> ::== The SMTP address of the journaled recipient

<redirection-type> ::== "Expanded" | "Forwarded"

<source-address> ::== The SMTP address of the recipient that was redirected to <recipient-address> by Categorizer

The following is a full list of the forms that the <recipient-specification> can take.

Note:
Ignore line-breaks in the examples. Recipient specifications are one line only.

Message was sent to user@contoso.com by To:

To: user@contoso.com

Message was sent to user@contoso.com by Cc:

Cc: user@contoso.com

Message was sent to user@contoso.com by Bcc.

Bcc: user@contoso.com

Message was originally sent by the client as To: to dl@contoso.com, which is a distribution group that was expanded by the categorizer to dl-member@contoso.com and possibly other recipients.

To: dl-member@contoso.com, Expanded: dl@contoso.com

Message was sent to user@contoso.com as Cc:, which was rewritten by the categorizer to fwd@contoso.com because an alternative recipient was configured.

Cc: fwd@contoso.com, Forwarded: user@contoso.com

Message was originally sent by the client as Bcc: to dl@contoso.com, which is a distribution group that was expanded to dl-member@contoso.com and possibly other recipients).

Bcc: dl-member@contoso.com, Expansion: dl@contoso.com

Message was sent to user@contoso.com as Bcc: which was rewritten by the categorizer to fwd@contoso.com because an alternative recipient was configured.

Bcc: fwd@contoso.com, Forwarded: user@contoso.com

Finally, if the envelope recipient is specified as follows, there is no information about whether this was To, Bcc, an expanded distribution group, alternative recipient, or so on.

Recipient: user@contoso.com

How Journal Report Formats Differ Between Exchange 2007 and Exchange 2003

Every line in the Exchange 2007 journal report is a (field, value) pair. In Exchange 2003, the recipients were listed by specifying the field “Recipients:” followed by all the recipients in the message, one recipient per line.

Unlike Exchange 2003, Exchange 2007 generates the report in its final form, instead of the “pre-journal report” format that Exchange 2003 creates. Therefore, Exchange 2007 can send journal reports directly to external repositories.

In Exchange 2007, the display name of the sender and recipients is no longer present in the report. In Exchange 2007, the categorizer does not resolve the display name of recipients. Therefore, this property is not available to Journaling.

For more information about journal reports, see Understanding Journal Reports.

Return to top

Error Handling

In Exchange 2007, if a temporary error, such as an error contacting Active Directory, is encountered during any phase of journaling, the message is put back into retry by the Journaling agent.

Journal reports are ordinary e-mail and so are susceptible to the same delivery problems that other e-mail messages may encounter. However, Journal reports are unique in that no organization can afford to lose a journal report. Therefore, journaling supports two modes of operation:

When a JournalingReportNdrTo address is configured, journal reports are sent with the return path in this address. This address is that of an alternate journaling mailbox. Therefore, NDRs of journal reports are sent to the alternate journaling mailbox. You should realize that if the JournalingReportNdrTo address is unavailable, the NDRs are deleted.
By default, JournalingReportNdrTo is set to the NULL SMTP address. The Microsoft Exchange Transport service checks whether a journal report has a NULL SMTP address. In that case, Transport puts the message in the retry queue instead of NDR it. Therefore, journal reports never NDR.
Note   If problems occur during delivery of a journal report, the following periodic events are logged as errors:
Event 9213   A non-expirable message that has the Internal Message ID %1 could not be categorized. This message may be a journal report or other system message. The message will remain in the queue until administrative action is taken to resolve the error. Other messages may also have encountered this error. To further diagnose the error, use the Queue Viewer or the Exchange Mail Flow Troubleshooter.
Event 8010   A non-expirable message could not be categorized. This message may be a journal report or other system message. The message will remain in the queue until administrative action is taken to resolve the error.

For more information about how to configure an alternate journaling mailbox, see How to Configure an Alternate Journaling Mailbox.

Return to top

Performance Monitoring

Performance counters provide a method to monitor the Journaling agent on each Hub Transport server. The following table lists the journaling performance counters that are available on each Hub Transport server.

Performance counter	Description
Journal Reports created Total	The total number of journal reports that were created.
Journal Reports Created/sec	Total number of Journal reports created per second.
Journal Processing Time	The total time spent executing journal rules and creating reports in milliseconds.
Journal Processing Time per Message	The average time in milliseconds spent in journaling per message. This includes the time spent executing journal rules and creating journal reports.
Messages Processed by Journaling	The number of messages processed by journaling including those that were not journaled.
Users Journaled	The number of message senders and recipients for which journal reports were generated.
Users Journaled/sec	The number of message senders and recipients for which journal reports were generated per sec

Note These performance counters are reset when the Microsoft Exchange Transport service is restarted.

Return to top

Conclusion

I hope this white paper has given you a glimpse into how powerful and versatile journaling in Exchange 2007 can be. From flexibility using per-recipient scoping to ease of administration by replicating journal rule configuration throughout your Exchange organization, Exchange 2007 helps you meet your compliance requirements for capturing e-mail communications.

Additional Information

For more information about transport journaling in Exchange 2007, see the following topics: