The Exchange message categorization system is made up of two components, a base categorizer and the Exchange categorizer. The base categorizer is made up of code originally implemented in Aqueue.dll. Exchange Server 2003 replaces Aqueue.dll by registering a DLL called Phatq.dll. Phatq.dll includes all the features available in Aqueue.dll, plus Exchange-specific features. The Exchange categorizer is implemented in PhatCat.dll. PhatCat.dll is registered for the events in the OnCategorize event category. The advanced queuing engine triggers these events through the base categorizer. The base categorizer and Exchange categorizer jointly process the message in ten categorizer events.

Note:
The base categorizer triggers the events that drive the message categorization process.

The advanced queuing engine triggers the following ten categorizer events:

• Register   The base categorizer and the Exchange categorizer use this event to initialize their interfaces. For example, the Exchange categorizer initializes its interfaces to Directory Service Access (DSAccess), routing engine, and store driver. If any one of these interfaces fail, then the Exchange categorizer fails to initialize itself.
  All categorizers require interfaces to these components for the following reasons:
  
  1. Communication with DSAccess is required to determine if recipients are in the DSAccess cache, in which case the required information can be obtained directly from DSAccess without the overhead of performing Active Directory lookups.
     The Exchange categorizer also determines the list of global catalog servers that DSAccess is using and provides this list to the base categorizer to use for its Lightweight Directory Access Protocol (LDAP) lookups. By default, the base categorizer uses the Active Directory domain topology to determine the global catalog servers for LDAP lookups. However, on a server running Exchange Server 2003, the categorizer should use the global catalog servers that DSAccess determined dynamically, based on the Active Directory site topology, or statically, if you hard coded global catalog servers in a DSAccess profile in the registry. For more information about the DSAccess discovery process, see Exchange Server 2003 and Active Directory.
     
  2. Communication with the routing engine is necessary, for example, when encapsulating a non-SMTP one-off address to an SMTP address. A one-off address is an address that does not resolve against a recipient object in Active Directory. The Exchange categorizer calls the routing engine to determine if a route to the target exists. If a route does not exist, the categorizer generates an NDR. If a route exists, the categorizer stamps the one-off recipient with the encapsulated address on the MailMsg object. Later, the advanced queuing engine passes the address information to the routing engine, which then determines the next hop for the recipient.
     

     Note:
     The MailMsg object is an in-memory structure that contains a message transfer envelope, plus a pointer to the actual message in either the NTFS store or the Exchange store. The message transfer envelope has all message header properties and recipient user information that the categorizer needs to process the message.

  3. Communication with the Exchange store driver is required in certain situations during categorization. For example, the Exchange categorizer might have to copy a message from the \Queue folder to enable the Exchange store to process and convert RFC 822 content using the content conversion logic that is implemented in the Internet mail (IMAIL) component of the Exchange store.
     
• BeginMessageCategorization   This event is called once for each MailMsg object submitted to the base categorizer, which signals the beginning of a message categorization process.
  
• EndMessageCategorization   This event is called for each MailMsg object submitted to the base categorizer. It signifies the end of message categorization.
  
• BuildQuery   This event is called once for each recipient and the sender that must be determined. However, query-based distribution group members are not determined, because their attributes are already determined when processing the query-based distribution group. For all recipients that must be determined, the Exchange categorizer verifies whether the recipient resides in the DSAccess cache. If this is true, the Exchange categorizer returns an ICategorizerItemAttributes object to bypass the base categorizer directory service lookup code. Processing continues with the ProcessItem event for this recipient. If the recipient is not in the DSAccess cache, the LDAP-lookup code of the base categorizer creates an LDAP-compliant search filter for the user, which is typically based on the proxyAddresses attribute and the input address, for example:
  
  (proxyAddresses=smtp:ted@fabrikam.com)

  The input address can be an SMTP address, an X.400 address, or a non-Exchange address, depending on the source and destination of the message.
  

  Tip:

  To illustrate how an LDAP filter, based on proxyAddresses, can be used to find a particular recipient object in Active Directory, you can use the LDIFDE tool (Ldifde.exe) included in Windows Server 2003. The LDIFDE command-line parameter "r" allows you to specify an LDAP search filter. For example, you can use the following command to find Ted in the global catalog on Server01 in a domain called fabrikam.com: ldifde -f c:\Ted.ldf -s Server01 -t 3268 -d "dc=fabrikam,dc=com" -p subtree -r "(proxyAddresses=smtp:ted@fabrikam.com)". If Ted has an SMTP address of Ted@fabrikam.com, LDIFDE finds the recipient object in Active Directory and writes all of its attributes to a plain-text file called Ted.ldf. The Exchange categorizer uses exactly the same principle to find recipient objects, retrieve default SMTP, X.400, and legacyExchangeDN attributes from the recipient, and set them as properties on the MailMsg object. Later, the Exchange categorizer uses this information in message processing.

  The Exchange categorizer uses the proxyAddresses attribute for almost all address types (except legacy Exchange distinguished names and X.500 distinguished names, because this address information is not stored in the proxyAddresses attribute). For example, when an Outlook user sends a message to another user in the Exchange organization or an external user that has a mail-enabled recipient object in Active Directory, the Outlook client specifies the legacyExchangeDN address of the recipient in the outgoing message for backward compatibility with Exchange Server 5.5. The Exchange categorizer then uses the legacyExchangeDN attribute in the search filter to find the recipient object in Active Directory.
  The Exchange categorizer must handle X.500 distinguished names when resolving the members of a distribution group, because the group members are listed with their distinguished names in the member attribute. In this situation, the Exchange categorizer parses the distinguished name to determine the relative distinguished name (RDN) of the recipient, which is the recipient's common name (CN) in Active Directory. The Exchange categorizer then uses the cn attribute in the search filter with the remainder of the distinguished name (which points to the recipient object's parent container in Active Directory) as the root of the LDAP search to find the recipient object. By default, the cn attribute is indexed in Active Directory, which results in an efficient directory lookup.
  
• BuildQueries   This event is called once for each batched lookup. The base categorizer uses this event to combine in a single query the individual searches for up to 20 recipients. Then SendQuery can issue a single search that returns a batch of recipients. It is usually more efficient to issue a single search for multiple recipients than to issue multiple searches for multiple recipients. This is true even though extra processing is required to handle a SortQueryResults event when performing searches in batches and matching the returned results with the individual recipients of the message. The matching required on results fewer than 20 is more efficient than requiring multiple round trip LDAP queries to Active Directory. The following is an example of an LDAP-compliant search filter that can be used to find multiple users at once:
  
  (|(proxyAddresses=smtp:Ted@fabrikam.com) (proxyAddresses=smtp:Birgit@fabrikam.com))

  Tip:

  To illustrate how an LDAP filter for multiple users works, you can use the following command to find Ted and Birgit in the global catalog on Server01 in a domain called fabrikam.com: ldifde -f c:\TedandBirgit.ldf -s Server01 -t 3268 -d "dc=fabrikam,dc=com" -p subtree -r "(|(proxyAddresses=smtp:Ted@fabrikam.com) (proxyAddresses=smtp:Birgit@fabrikam.com))". If Ted has an SMTP address of Ted@fabrikam.com and Birgit has an SMTP address of Birgit@fabrikam.com, LDIFDE finds both recipient objects in Active Directory and writes all of their attributes in separate sections to a plain-text file called TedandBirgit.ldf. The categorizer uses exactly the same principle to find multiple recipient objects.

• SendQuery   The categorizer triggers this event once for each batched lookup and then performs the directory search asynchronously.
  
• SortQueryResult   The categorizer calls this event once for each batched lookup and then determines which users belong to which directory object (the LDAP results returned for the query). The proxyAddresses attribute is used for matching results from lookups, based on an SMTP address, an X.400 address, or a non-Exchange address. The legacyExchangeDN attribute is used to match legacyExchangeDN lookups, and the distinguishedName attribute is used to match X.500 distinguished name lookups. The address information must uniquely identify each recipient for this to work. If values are not distinguishing, a 5.1.4 NDR results. The following table provides the details for the 5.1.4 NDR.
  

  Details of NDRs with code 5.1.4

  Numerical code	5.1.4
  Possible cause	Two objects have the same (proxy) address, and mail is sent to that address.
  Troubleshooting	Check the recipient address and resend the message. This issue can also occur if the recipient does not exist on the remote server.
  Comments	For more information about NDR codes, see Microsoft Knowledge Base article 284204, "Delivery status notifications in Exchange Server and in Small Business Server."

• ProcessItem   The base categorizer triggers this event to add the default SMTP, X.400, X.500, and legacyExchangeDN addresses for each recipient to the MailMsg object. The addresses that the Exchange categorizer sets on the recipient are based on the attributes returned for the recipient from Active Directory. The mail attribute contains the SMTP address, the textEncodedORAddress attribute contains the X.400 address, the distinguishedName contains the X.500 address, and the legacyExchangeDN attribute contains the legacy Exchange address.
  

  Note:
  The addresses used for the recipient after this point do not necessarily match the address used to look up the recipient in Active Directory. For example, a non-Exchange user might send a message to the secondary proxy address of an Exchange user. During the ProcessItem event, the Exchange categorizer replaces the secondary proxy address with the primary address of the Exchange user.

  The Exchange categorizer also has special code to handle contacts and one-off addresses. Contacts are non-Exchange recipients, but they reside in Active Directory. One-off addresses do not exist in Active Directory. The sender might have typed the recipient address directly in the To line or might have specified a contact object from his or her personal Contacts folder in Outlook. In either scenario, only the target address of the recipient is known and added to the MailMsg object. If a contact address or a one-off SMTP address matches an address space of the local Exchange organization, a conflict occurs, because contacts and one-off addresses must refer to recipients outside of the local Exchange organization.
  The categorizer enforces its authority for addresses that match address spaces specified in recipient policies when you select the This Exchange Organization is responsible for all mail delivery to this address check box. If a user sends a message to an address that is not in Active Directory but matches an address space in an authoritative recipient policy, the Exchange categorizer sets an error status on the recipient that later causes the advanced queuing engine to generate a 5.1.1 non-delivery report, which signifies an unknown address. The Exchange categorizer also considers itself authoritative for legacyExchangeDN and X.400 addresses that match the site address space of the local administrative group.
  

  Note:
  If you have an alternate server configured on an SMTP virtual server (the Forward all mail with unresolved recipients to host setting), the categorizer reroutes messages to non-existent addresses in its authoritative address spaces to the alternate server, instead of generating a 5.1.1 non-delivery report for the recipients. This action is taken during the CompleteItem event.

  The following table provides the details for the 5.1.1 non-delivery report.
  

  Details of non-delivery reports with code 5.1.1

  Numerical code	5.1.1
  Possible cause	The e-mail account does not exist at the organization to which this message was sent. For example, if an Internet user sends a message to user_does_not_exist@fabrikam.com, where your server is authoritative for fabrikam.com and no one in Active Directory has that address, a 5.1.1 NDR is generated. A 5.1.1 NDR is an "authoritative not found" NDR. It applies to SMTP addresses according to recipient policies, to legacyExchangeDN recipients according to the legacyExchangeDN attribute of the local administrative group, and to X.400 addresses according to the administrative group's X.400 site address. Otherwise, a 5.1.2 NDR is generated any time you have a non-SMTP address that is not routable and does not match a recipient object in Active Directory.
  Troubleshooting	Either the recipient address is incorrectly formatted or the categorizer cannot properly resolve the recipient. You must check the recipient address and resend the message to resolve this error. For more information about NDR codes, see Microsoft Knowledge Base article 284204, "Delivery status notifications in Exchange Server and in Small Business Server."

  For addresses that are not in Active Directory and do not match an authoritative recipient policy or local site address space, the categorizer does not register an error. Instead, it registers a relay to a remote destination.
  
• ExpandItem   The categorizer triggers this event to handle the following tasks:
  
  • Distribution list expansion   If the local server is an expansion server for the distribution groups in the message, the distribution groups can be expanded locally. The msExchExpansionServerName attribute lists the server running Exchange Server 2003 that is responsible for distribution list expansion. If this attribute is not set, any server running Exchange can expand that distribution group. If a server is specified and it is not the local server, the categorizer must forward the message to that server, because the mail-enabled group must be expanded on that specific server. When a distribution group is expanded, the categorizer resolves each individual recipient.
    
  • Restriction checking   The categorizer checks for any per-sender and per-recipient restrictions, limits, and message format settings and marks each recipient appropriately. For example, if the sender has a recipient limit and a message exceeds this limit, the categorizer requests the advanced queuing engine to generate a 5.5.3 NDR to ensure that no message has more than the allowed number of recipients.
    

    Details of NDRs with code 5.5.3

    Numerical code	5.5.3
    Possible cause	Too many recipients on the sent message.
    Troubleshooting	The recipient limit is a configurable limit on the receiving server. To resolve this issue, either increase the recipient limit, or divide the message into multiple messages to fit the server limit.
    Comments	For more information about NDR codes, see Microsoft Knowledge Base article 284204, "Delivery status notifications in Exchange Server and in Small Business Server."

  • Alternate recipients   Using Active Directory Users and Computers, you can specify alternate recipients for Exchange users on the Exchange - General tab under Delivery Options. The Exchange categorizer adds this recipient information to the MailMsg object and forwards the message. When transferring the message across an Exchange organization, a flag is passed in the message transfer envelope through XEXCH50 with the original recipient. This prevents multiple copies of a message from being forwarded to an alternate recipient. When the Exchange categorizer discovers this flag for an original recipient, it skips the code to add the alternate recipient.
    The Exchange categorizer also checks for loop conditions (for example, Ted forwards to Birgit and then Birgit forwards to Ted). If a loop is detected, the categorizer informs the advanced queuing engine to generate a 5.4.6 NDR for the first user in the loop.
    

    Details of non-delivery reports with code 5.4.6

    Numerical code	5.4.6
    Possible cause	A categorizer forward loop is detected. A targetAddress attribute is set on a mailbox-enabled user that contains an address that is in an authoritative SMTP domain. The targetAddress attribute must point to a remote domain. A 5.4.6 NDR is also generated when you have a forwarding loop in Active Directory. For example, this NDR is generated if a chain of Exchange users has alternate recipients that point to each other circularly.
    Troubleshooting	Check the contact's alternate recipient. Check and remove the targetAddress attribute from mailbox-enabled users.
    Comments	For more information about NDR codes, see Microsoft Knowledge Base article 284204, "Delivery status notifications in Exchange Server and in Small Business Server."

  • Message bifurcation   If recipients require different message formats, the Exchange categorizer bifurcates the message to create multiple message copies. Message bifurcation is explained in more detail later in this section.
    
• CompleteItem   The base categorizer calls this event to perform final processing when the work of all other categorizer event sinks is complete. Final processing includes the following tasks:
  
  • Journaling   Journaling is the process of copying messages sent or received by users on an Exchange server to a message archive. If the current server is the first server to handle a sender or recipient for which journaling must occur (for example, because message journaling is enabled for the sender's or a recipient's local mailbox store), the Exchange categorizer adds the mailbox store's journaling address to the message transport envelope and transfers the message.
    The journaling configuration is stored in Active Directory and is available to all Exchange servers in the organization. For example, an Exchange user (whose home mailbox store has journaling enabled) might use an Internet client to send messages through SMTP to a bridgehead server that is part of the Exchange organization. The bridgehead server then adds the journaling address to the message transfer envelope and forwards a copy of the message to the archive mailbox for that sender. Servers running Exchange Server 2003 communicate message transfer envelope information, including the list of journal addresses for sender and recipients, using the XEXCH50 command. If multiple Exchange servers are involved in message transfer and a server finds a journaling address in the message transfer envelope, it does not add the journaling address again, thus preventing duplicate messages in the message archive.
    

    Note:

    Message journaling might not reliably account for blind carbon-copy recipients, recipients from transport forwarding rules, or recipients from distribution group expansions. If you must record the message transport envelope data, in addition to the message data, you must enable Envelope Journaling. This is accomplished using the E-Mail Journaling Advanced Configuration tool, available for download at the Exchange Server Email Journaling Advanced Configuration Web site.

  • Forwarding unresolved recipients   If you configured the SMTP virtual server to forward all messages with unresolved recipients to an alternate host, the Exchange categorizer sets the destination server for all unresolved recipients to the fully qualified domain name (FQDN) of the alternate server.
    
  • Recipient marking   The Exchange categorizer marks each recipient by setting a per-recipient property on the message. This property indicates the destination server for each recipient. The typical format of this property is the FQDN of the recipient's home server. Based on the FQDN that the categorizer sets on each recipient, the advanced queuing engine decides whether to deliver the message locally or call the routing engine. All messages matching the local server's FQDN go straight from the pre-routing queue to the local delivery queue, without performing message routing. For recipients that are not local, the advanced queuing engine must call the routing engine later to determine the next hop for message transfer.
    

    Note:

    The Exchange categorizer determines the FQDN of each recipient's home server through a component called MDAccess. MDAccess uses the configuration information in Active Directory to determine the network address of the server hosting a recipient's mailbox store. The Exchange categorizer sets the recipient's homeMDB attribute as a property on the recipient in the message transfer envelope, if the recipient's mailbox resides on the local Exchange server. With this information the Exchange store driver later determines the mailbox store to which it delivers the recipient's message.

  • Redirecting delivery status notifications   When a user sends a message on behalf of a distribution group (that is, the user specifies the group in From) and requests delivery or read receipts, the delivery status notifications are generated and sent to the distribution group, rather than to the sender only. To address this issue, the categorizer directs status notifications to the actual sender by changing the sender address in the message transfer envelope of the message to the address of the actual sender.
    Users rarely send messages on behalf of a distribution group. A more common situation occurs when a user receives multiple out-of-office notifications, auto replies, and delivery status notifications, such as non-delivery reports, after sending a message to a large distribution group. To mitigate this issue, the Exchange categorizer suppresses out-of-office notifications and auto replies by default when sending messages to a distribution group. To accomplish this, the Exchange categorizer adds a property to the recipients in the message envelope. The Exchange store uses this property as a parameter during local delivery operation to suppress rules which would otherwise generate out-of-office notifications and auto replies. This extended property of the message envelope is transmitted between servers running Exchange Server 2003 using XEXCH50, if the individual distribution group members reside on different mailbox servers.
    The Exchange categorizer redirects or deletes out-of-office notifications, auto replies, and delivery status notifications according to the criteria listed in the following table.
    

    Delivery status notification redirection and deletion criteria

    Criteria	Action
    Distribution group has an owner, and delivery reports are configured to be sent to this owner	Redirect non-delivery reports to notify the configured owner of the group when an error occurs during the delivery of a message to the group or to one of its members. The categorizer redirects any non-delivery reports for group members that would usually return to the message sender. By default, only NDRs are redirected to the group owner. In cases in which a sender requests explicit status notifications (such as a delivery receipt notification), SMTP protocol delivery status notification extensions are used to generate the reports. If a sender requests a delivery status notification in a message to a group with report redirection enabled, the sender receives the delivery status notification when the group is either expanded successfully or redirected to its expansion server (if the local server is not an expansion server for the group).
    Delivery reports are configured to be sent to message originator	Send delivery status notification to the sender of the message. The categorizer suppresses out-of-office notifications and auto replies if the Send out-of-office messages to originator check box, on the Exchange-Advanced tab, is not selected for the group. By default, this check box is not selected.
    Distribution group is configured to suppress delivery reports for group members to avoid disclosing membership information	Suppress all out-of-office notifications, auto replies, and all types of delivery status notifications (such as delivery notifications and read receipts). This is similar to redirecting NDRs, except that the categorizer suppresses all types of delivery notifications, including non-delivery reports for individual group members. To block all delivery reports, the categorizer changes the sender address in the message transfer envelope of the message to "<>". If a sender requests a delivery status notification (such as a delivery receipt notification) in a message to the group, SMTP protocol delivery status notification extensions generate the delivery status notification when the group is either expanded successfully or redirected to its expansion server (if the local server is not an expansion server for the group).

    Note:

    By default, the categorizer suppresses out-of-office notifications, auto replies, and delivery status notifications when a user sends a message to a distribution list. You can configure this by selecting the Send out-of-office messages to originator check box, on the Exchange-Advanced tab, in the distribution group properties. For more information, see Microsoft Knowledge Base article 325469, "Automatic replies from distribution group do not work in Exchange Server 2003 or Exchange 2000 Server."

  • Status code mapping   The Exchange categorizer also maps the status codes that previous categorizer sinks have returned to the recipients in the e-mail message. Error codes then cause the advanced queuing engine to generate NDRs for affected recipients.
    

As mentioned previously, the categorizer might create multiple copies of a message during the categorization process. This process is called bifurcation and is performed any time different recipients must receive different copies of the same message. Bifurcation is performed for the following reasons:

• Content conversion   Content conversion must occur whenever an Exchange user sends a message in MAPI format to a non-MAPI user. For example, an Outlook user might send a message to a recipient on the Internet, in which case the categorizer must perform content conversion from MAPI to an Internet message format. Content conversion must also occur when a MAPI message must be transferred over an SMTP-based routing group connector in a mixed-mode Exchange organization. Bifurcation is required for content conversion because the categorizer cannot change the content of existing messages. You can read more about content conversion later in this section.
  
• Removing delivery and read receipt requests on a per-recipient basis   This is necessary when a message with a read receipt request is sent to a distribution group with hidden membership information and an additional individual recipient in the To line. Because the membership of the group must remain confidential, read receipts must not be generated when the distribution group members receive the message. Otherwise, the sender would be able to identify the group members based on the read receipts received. To avoid this, two copies of the message are created, one for the distribution group with hidden membership and the other for the individual recipient. The read receipt request will be removed from the message to the distribution group.
  
• Delivery status notifications with multiple recipients   When the Exchange categorizer detects delivery status notifications with multiple recipients, the Exchange categorizer bifurcates the message for each recipient, because the Exchange MTA (to comply with the X.400 standard) does not support more than one recipient per notification. Because the categorizer cannot determine if the MTA is involved in message transfer (only the routing engine can determine this), the categorizer must create a separate delivery status notification for each individual recipient.
  

Bifurcation occurs on the server running Exchange Server when the message is submitted by the client. You can check the number of new messages created by the categorizer using the Performance tool. The following figure illustrates the relevant performance counter.

The Cat: Messages bifurcated performance counter

Users of MAPI clients, such as Outlook, can specify on a per-message or per-recipient basis whether to send the message in rich-text, HTML, or plain text format. The categorizer must convert the message accordingly. Administrators can also specify preferred formats in the properties of mail-enabled recipient objects in Active Directory or define Internet mail formats on an address space basis (for example, per Internet domain, in Exchange System Manager, under Global Settings). Thus, messages sent to users in an Internet domain can be formatted in rich-text, Multipurpose Internet Mail Extensions (MIME), or Unix-to-Unix encoded (UUEncoded). The categorizer uses specific logic to coalesce these various format settings for each recipient. When the categorizer resolves the message recipients, it might discover that different message formats are required for subsets of recipients or individual recipients. The categorizer must then bifurcate the message to have the correct message format, such as rich-text, HTML, or plain text, for each recipient.

Content conversion is also required for MAPI messages to Exchange recipients inside the local Exchange organization, if the messages are transferred over SMTP. Servers running Exchange Server 2003 in the local routing group always communicate with each other over SMTP. Servers running Exchange Server 2003 in different routing groups communicate over SMTP when the Routing Group connector or SMTP connectors are deployed. To support SMTP, the MAPI format must be converted to RFC 822 format so that the message can be transferred.

Note:
Content conversion is done on the server where the user submits the message. This allows the message to move from server to server by means of SMTP, without further conversion. Content conversion is performed for MAPI messages only.

The message conversion process in Exchange Server 2003 is called IMAIL. IMAIL is an internal component of the Exchange store. The Exchange categorizer does not perform the actual message conversion. The Exchange categorizer creates copies of the message using the Exchange store driver only and sets some properties in the message transfer envelope of each copy. The store driver then uses these property settings as parameters to specify which format to request when requesting the RFC 822 content from the Exchange store. The Exchange store uses IMAIL to convert the MAPI message to RFC 822 format, using the requested formatting parameters. When producing the RFC 822 content of a message, IMAIL produces a rendering of the MAPI message only. The actual message in the Exchange store is not changed. Changes to the rendered RFC 822 content are not dynamically correlated back to the MAPI message that was used to produce the RFC 822 content.

Exchange Server 2003 uses transport neutral encapsulation format (TNEF) to convert MAPI messages to RFC 822 format. TNEF appears on a message as a MIME attachment of type application/ms-tnef. The attachment is called Winmail.dat. It contains the full message content and any attached files. Only MAPI clients, such as Outlook, are able to decode the Winmail.dat attachment. Non-MAPI clients are unable to decode TNEF and might show Winmail.dat as a typical, but useless file.

Note:
Recipients with mailboxes on an Exchange server, who use Internet clients to access their messages, are not considered non-MAPI recipients. This is because the Exchange store that hosts the mailboxes can produce the necessary RFC 822 content in non-MAPI format when users download MAPI messages from their Inboxes using a POP3 or an IMAP4 client.

There are several possible Exchange-to-Exchange transfer scenarios that require MAPI to RFC 822 conversion:

• Recipient is on an Exchange server in the same routing group   Exchange Server 2003 renders MAPI messages into Summary-TNEF (S/TNEF) format, a special kind of transport-neutral encapsulation format (TNEF), which has no plain text part and is rendered in eight-bit binary format. S/TNEF messages consist only of Winmail.dat.
  

  Note:

  Within a routing group, Exchange Server 2003 always uses S/TNEF, because in all remote delivery cases, the message is guaranteed to take either an SMTP hop directly to a server running Exchange 2000 Server or Exchange Server 2003 or go to the Exchange MTA. Exchange 2000 Server and Exchange Server 2003 support binary MIME. On the other hand, if the message is passed to the Exchange MTA for delivery to a server running Exchange Server 5.5 through RPCs, message conversion is not required, because the RFC 822 format is not used.

• Recipient is on an Exchange server in another routing group and the Exchange organization is operating in native mode   Exchange Server 2003 renders MAPI messages into Summary-TNEF (S/TNEF) format, because in native mode an Exchange organization can include only servers running Exchange 2000 Server and Exchange Server 2003 that support binary MIME.
  
• Recipient is on an Exchange server in another routing group and the Exchange organization is operating in mixed mode   In mixed mode, it is possible to use the Internet Mail Service of Exchange Server 5.5 as an SMTP connector, but Internet Mail Service does not support binary MIME. Because the RFC 822 representation of S/TNEF that IMAIL generates is binary MIME, Internet Mail Service is unable to transfer S/TNEF messages. Because the Exchange categorizer cannot detect beforehand what route a message will take, the categorizer does not convert the message to S/TNEF for a recipient that is on a server outside the local routing group in mixed mode. To accommodate a potential Internet Mail Service instance in the transfer path, the Exchange categorizer converts the message to a plain text part and an attachment in legacy TNEF, which is seven-bit MIME that Internet Mail Service can transfer.
  
• Recipient is a MAPI recipient outside the local Exchange organization    Users and administrators can enable the transfer of TNEF across the boundaries of the local Exchange organization for recipients in external messaging environments who use Outlook. Because the recipient is not in the local Exchange organization, the Exchange categorizer cannot determine if all SMTP hosts involved in the message transfer support binary MIME. Correspondingly, the Exchange categorizer converts the message to a plain text part and an attachment in legacy TNEF.
  

  Note:
  Non-MAPI recipients typically prefer to receive a message in plain text or HTML without TNEF, because their clients cannot decode the Winmail.dat file that includes the message and all attachments. TNEF encapsulation prevents non-MAPI clients from accessing attachments. Non-Microsoft tools, such as EPOC WMDecode for Windows, might be able to extract attachments from Winmail.dat files.

• MAPI messages sent to public folders   Messages sent to public folders are always relayed in legacy TNEF format. You can read more about public folder message handling later in this section.
  
• MAPI messages sent to an expansion server over SMTP   If a message contains a distribution list with an explicit expansion server that is not the local server, the message is forwarded to the expansion server in legacy TNEF format (if SMTP is used for message transfer). In this case, a property is transmitted in the message transfer envelope through XEXCH50 that notifies the expansion server of the time the message was originally received through the Exchange store driver. After the categorizer on the expansion server expands the distribution list, it must apply the effective RFC 822 message formats to the individual recipients. The categorizer uses the Exchange store driver to copy the message to the Exchange store, where IMAIL reads the TNEF data to construct a MAPI message with the submit time of the original message. The SMTP transport subsystem can then read the MAPI message from the store in an RFC 822 format consistent with the recipients' formatting requirements.
  

You can control the TNEF format behavior for sending messages by adding the following registry key. The number nn represents the virtual server instance for this machine.

Public folders can be mail-enabled so that users can send messages to them. However, unlike mailbox-enabled user accounts, which can have their mailboxes only on one designated Exchange server in an organization, public folders can be replicated between multiple servers and can be missing from other servers that have a public folder store associated with the public folder's top-level hierarchy. Therefore, it is difficult to determine the delivery location for messages sent to the public folder.

To perform message delivery, the Exchange categorizer must deliver the message to a public folder store that knows where the replicas of the public folder reside. This information is contained in the Exchange store in the PTagReplicaList attribute. Only the Exchange store with a public folder store that is associated with the public folder's top-level hierarchy has this PTagReplicaList information.

To find a public folder store that is associated with the public folder's top-level hierarchy, the Exchange categorizer reads the public folder's homeMDB attribute. The homeMDB attribute contains the distinguished name (DN) of the top-level hierarchy object in Active Directory. This object, in turn, has an msExchOwningPFTreeBL attribute that lists the public folder stores associated with the top-level hierarchy. The Exchange categorizer then chooses a public folder store from that list and directs the message to that public folder store. The public folder store determines the PTagReplicaList entry for that folder, readdresses the message to a public folder store that holds a replica of the public folder, and resubmits the message. The message again goes through the advanced queuing engine, including categorization. When the Exchange categorizer locates the updated folder location in the public folder store, it sets the updated folder location as the destination of the message and re-routes the message.

The Exchange categorizer uses the following prioritization, from top to bottom, to select the public folder store for the message:

1. Public folder stores on the local server running Exchange Server have the highest priority
   
2. Public folder stores in the local routing group
   
3. Public folder stores in the local administrative group
   
4. Public folder stores in the local Exchange organization
   
5. Public folder stores on servers running Exchange Server 5.5
   

Note:
If multiple public folder stores exist in the local routing group, the Exchange categorizer chooses the first from the list.

As mentioned in the section "Exchange Categorizer" earlier in this topic, the Exchange categorizer determines the list of global catalog servers to use for LDAP lookups from DSAccess. DSAccess determines this list dynamically, based on the Active Directory site topology or statically, if you hard coded global catalog servers in a DSAccess profile in the registry, as explained in Exchange Server 2003 and Active Directory.

By default, DSAccess determines the list of global catalog servers dynamically, which enables global catalog servers to be excluded from the list, if they become unavailable for any reason. The connection retry period for an unavailable global catalog server is five minutes. By default, DSAccess recalculates its list at least every 15 minutes. The Exchange categorizer calls DSAccess once every hour to update the list of global catalog servers.

The Exchange categorizer also updates the list of global catalog servers if there are more than 100 connection failures per global catalog server. A global catalog server might be down for maintenance or other reasons, or there might be a network communication problem causing an LDAP connection to fail. You can use the following registry parameters to specify the timeout values that the categorizer uses to classify LDAP connections as not operational.

Caution:
Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.

If a categorization fails because MaxConnectionRetries is exceeded, the categorizer places the affected message back in the pre-categorization queue and informs the advanced queuing engine that the categorization might be successful in a later attempt. By default, the advanced queuing engine then retries the categorization after one hour. You can adjust this time period using the following registry parameter.

If multiple global catalog servers are available to a server running Exchange Server 2003, the Exchange categorizer can load balance LDAP searches between all of these global catalogs. By default, the Exchange categorizer starts load balancing LDAP searches when more than 16 LDAP searches are pending on one global catalog server. The Exchange categorizer chooses global catalog servers based on cost values. The cost values are determined by the following characteristics:

• Number of existing LDAP connections   The Exchange categorizer prefers existing LDAP connections over new connections, because it is more efficient to use a connection that is already established than to establish a new connection to a global catalog server. Establishing connections imposes processing overhead.
  By default, the base categorizer can open up to eight (depending on the workload) concurrent LDAP connections per global catalog server to perform directory lookups. You can adjust the number of connections using the following registry key.
  

  Location	HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Parameters
  Value	MaxLdapConnections
  Type	REG_DWORD
  Value Data	0x8
  Description	This is the maximum number of LDAP connections that the components in the SMTP service can open to a global catalog server. The default is eight connections. Note: This value applies individually to each process in the categorizer that performs LDAP lookups: one resolves recipients on submitted messages during categorization and one checks message restrictions per recipient. Each of these processes can have eight connections, thus the maximum total is 16 connections to global catalog servers.

• State of existing LDAP connections   The Exchange categorizer prefers available LDAP connections that have no pending searches.
  
• Locality to the Exchange server   The Exchange categorizer prefers global catalog servers in the local Active Directory site over global catalog servers in remote sites. It is assumed that TCP/IP connections in the local site are fast and reliable.
  
• Number of LDAP pending queries   The categorizer prefers idle connections over connections with pending queries. If all connections have pending queries, the Exchange categorizer chooses the connection with the least number of pending queries.
  

The Exchange categorizer selects the global catalog server with the lowest cost. If multiple global catalog servers have the same cost value, the categorizer performs load balancing across all available LDAP connections in a round robin fashion. The Exchange categorizer selects a connection as follows:

1. The Exchange categorizer iterates through the list of existing LDAP connections and automatically selects the first connection that has no pending searches.
   
2. If no LDAP connection exists or is available, the Exchange categorizer establishes a new connection to the global catalog server.
   

The Exchange categorizer can perform lookups asynchronously and in batches of up to 20 recipients. Performing a single Active Directory lookup for multiple recipient objects using an LDAP OR filter can result in a performance gain, even though some extra processing is required to match the results to the recipients in the message. Batched LDAP searches work well for individual recipient objects that can be grouped together into a single query to a global catalog. Most Exchange categorizer operations are in this category, but there are exceptions.

The categorizer uses non-batched queries in the following situations:

• The categorizer must expand a dynamic distribution group.
  
• The categorizer must request paged attributes. For example, this is the case for distribution groups with more than 1,000 members.
  

Note:
The Exchange categorizer checks with DSAccess to determine if a recipient object is in the DSAccess cache. For cached objects, directory lookups are not performed.

You can manage the performance of the Exchange categorizer using the following registry keys. For example, you can increase the maximum number of recipients in a batched search. However, dramatically increasing this number might actually have a negative impact on performance, because the overhead for matching results to recipients also increases. In general, the default values are sufficient for most situations. Therefore, it is not a good idea to change these values without the assistance of a Microsoft Product Support specialist.

Messages are categorized only once. For messages in the \Queue folder on the file system, the categorizer uses alternate data streams, a little known NTFS feature, to persist the MailMsg property stream, which includes message envelope and categorization information. Alternate data streams enable data storage in hidden files, which are linked to a visible file on an NTFS partition. When the SMTP service cannot transfer a message immediately and must retry at a later time, the message is saved and closed. Part of that operation involves saving the existing MailMsg property stream, so that it can be reloaded and used when the message transfer is retried. However, if you must categorize a message again (for example, if it is queued for a destination server that no longer exists) you will notice that categorization is not performed a second time.

The advanced queuing engine stores messages either as .eml files in the SMTP virtual server's \Queue directory or as Exchange installable file system (ExIFS) files in the Exchange store. For .eml files, the categorizer uses two NTFS alternate data streams, named PROPERTIES and PROPERTIES-LIVE, during the categorization process to persist the properties of the MailMsg object and categorization information. The PROPERTIES data stream provides a copy of the original message. The PROPERTIES-LIVE data stream provides a copy of the current message. The alternate data streams are removed when the SMTP service moves a message to the \Badmail folder. In this situation, the message is saved with a .bad file name extension, and the property stream is saved in a separate file with a matching file name, but with a .bdp file name extension.

Note:

The way that the property stream is saved depends on the store driver that is used. The NTFS store driver saves property streams using alternate data streams. The Exchange store driver saves property streams by copying the data to a special binary MAPI property of the message (if the property stream is small) or to a separate ExIFS file (if the property stream is large), in which case a link to the ExIFS file is saved in the binary MAPI property.

You can use Notepad to view the alternate data streams for an .eml file in the SMTP virtual server's \Queue directory. For example, the following command shows the categorization information for a test message with the file name NTFS_0ec25fe701c4120a00000001.EML: notepad C:\NTFS_0ec25fe701c4120a00000001.EML:PROPERTIES-LIVE. Note that the period at the end belongs to the command. If you omit it, Notepad will append a .txt filename extension to PROPERTIES-LIVE, but PROPERTIES-LIVE.txt is not the name of the alternate data stream. The first part of the file name points to the parent file of the stream. The second part is the actual stream name.

The following figure shows sample content from the parent file (on the left), together with MailMsg property information in the alternate data stream (on the right). Note that the PROPERTIES-LIVE stream in the top window contains detailed recipient information obtained for sender and recipient from Active Directory.

A message file with an alternate data stream for categorization information

Note:

If you move an NTFS file with an alternate data stream to a File Allocation Table (FAT) partition, the alternate data stream is lost, because FAT does not support this feature. This loss includes categorization information and message transfer envelope. Later, if you move this file to the pickup directory, the P2 (RFC 822) recipient list is used to deliver the message, unless x-sender and x-receiver headers are specified. This list might differ from the P1 (RFC 821) recipient list that was used to send the message originally, so some recipients might receive the message twice, Bcc'ed recipients might not receive the message at all, or unintended recipients might receive a copy of the message. These outcomes occur because the original P1 recipient list was lost with the MailMsg property stream, leaving the SMTP service with no information other than the P2 recipient list. The P2 recipient list, however, is not designed to maintain a list of recipients for the purpose of transport and delivery.

The previous discussion demonstrates that it is not a good idea to move files to a FAT partition to drop the MailMsg property stream, thus forcing the transport subsystem to re-categorize a message.

You might have to force re-categorization of a message in the following situations:

• Metabase corruption   If the Internet Information Services (IIS) metabase is corrupt or replaced with a non-Exchange version, messages might be categorized by the wrong categorizer version. To categorize the messages using the Exchange categorizer (perhaps after reinstalling Exchange Server 2003), you must re-categorize the messages.
  
• Incorrect expansion server   If you change the expansion server for a distribution list, the wrong distribution list expansion server might be stamped on the message object until the distribution list changes are replicated across Active Directory. If this occurs, the message is sent to the incorrect expansion server. For example, if the expansion server is removed from the network and messages are already categorized on the local server, you must re-categorize the messages to send them to the correct expansion server.
  
• Incorrect back-end server   A categorization issue might also be caused if you restore mailboxes on an Exchange server other than the original server in the organization. For example, you might decide to restore mailboxes on a different server if the hardware of a mailbox server fails. Any existing messages in queues on other servers running Exchange Server (such as front-end servers) might not be delivered, because the advanced queuing engine attempts delivery to the non-existing server. Unless you re-categorize the messages, the previous server's FQDN is contained in the message transfer envelope.
  

In these situations, you must re-categorize the messages. However, server restarts do not affect alternate data streams. For this reason, restating the server running Exchange Server does not result in the re-categorization of messages. To trigger a re-categorization without moving files to a FAT partition, you must reset the message status using the following registry key:

Caution:
Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.