The Advanced Queuing Engine

 

Tópico modificado em: 2006-08-17

The advanced queuing engine is a key component in Exchange 2003 message handling, because all messages must pass through this engine, even when sender and recipient are located on the same server running Exchange Server 2003. This enables Exchange Server 2003 transport components to process every message. No e-mail message can bypass the transport subsystem.

Dica

The base SMTP service of Windows Server 2003 uses an advanced queuing engine implemented in Aqueue.dll to process received messages. The extended version of the SMTP service does not use Aqueue.dll. Exchange Server 2003 replaces this component with an Exchange advanced queuing engine, implemented in Phatq.dll. To load Phatq.dll, Exchange Server 2003 modifies the SmtpAdvQueueDll property for the SMTP service in the IIS metabase and points it to the Exchange advanced queuing engine. Aqueue.dll and Phatq.dll perform similar functions, but Phatq.dll is the only version that works with Exchange Server 2003.

Events Triggered by the Advanced Queuing Engine

As the following figure illustrates, the advanced queuing engine acts as an information controller or dispatcher of system, store, and transport events to process each message after it reaches the SMTP transport subsystem.

19d38f34-9e8f-430e-8194-c7cfd15dce53

The advanced queuing engine dispatches the following events:

  • SMTP Transport OnSubmission   This event occurs when a message arrives at the transport subsystem through \Pickup directory, SMTP connection, or Exchange store driver. For categorization, routing, and delivery, the message must pass to the advanced queuing engine. This action triggers an OnSubmission event, also named OnTransportSubmission. The advanced queuing engine uses this event to invoke event sinks, such as an anti-virus scanner, that check all incoming messages before any other transport processing occurs. Accordingly, for example, the Exchange Transport AntiVirus API event sink is registered for the OnSubmission event. Another sink registered for this event is the Exchange Transport XEXCH50 Submission sink, which prepares messages with XEXCH50 envelope data for further processing from remote servers running Exchange Server 2003.

    Dica

    The OnSubmission event is the only event that offers a Collaboration Data Objects (CDO) interface. Therefore, you can program event sinks using Microsoft Visual Basic or Visual Basic Scripting Edition (VBScript). You must program all other event sinks using C/C++ or Microsoft Visual C#.

    CDO-based event sinks can register for the CDO_OnArrival event, which is a wrapper around the OnSubmission event that provides a handle to the message in CDO message format. The major benefit to using CDO_OnArrival is that the CDO message object interface has many useful methods, such as the parsing of MIME and Request for Comments (RFC) 822 headers. For details about extending the SMTP service through a CDO sink, see Microsoft Knowledge Base article 837851, "How to configure an Internet Information Services SMTP virtual server to archive or to remove messages in an Exchange Server 2003 test environment."
    The major drawback of CDO-based event sinks is that the CDO interface adds overhead. VBScript-based event sinks do not perform as fast as sinks written in C, C++, or Visual C#. It should also be noted that CDO-based event sinks operate synchronously, and there is a time limit of 60 seconds to process the message. After 60 seconds, the advanced queuing engine assumes that the script is not responding and stops the processing. The 60 second limit is hard coded and cannot be changed. Moreover, the CDO interface does not support changing the content of store-submitted messages. This is a limitation that CDO_OnArrival event sinks share with all other event sinks. This limitation exists because Exchange converts a store-submitted message to a temporary SMTP version for the event sink to handle, and then discards the temporary version after the sink finishes processing. For more information about this issue, see Microsoft Knowledge Base article 273233, "PRB: CDOEX: Cannot Change MAPI Message Contents in a CDO SMTP Event Sink."
    Because Exchange discards the temporary copy of a store-submitted message, you cannot use an event sink to add a disclaimer or other modifications to all outbound messages, unless you force all messages to be received through SMTP. To do this, you must install the event sink on a bridgehead server that is separate from the mailbox servers in your organization. Servers running Exchange Server 2003 communicate with each other through SMTP, so all messages transferred to the bridgehead arrive through SMTP.

  • SMTP Transport OnPreCategorize   This event occurs immediately before a message is sent to the categorizer. This event is similar to the OnSubmission event, except that it offers no CDO version. By default, no sink is registered for this event.

  • SMTP Transport OnCategorize   This event occurs when a message must be categorized. This is not a single event, but an event category. There are ten different kinds of events that can be triggered for sinks that bind to this category. An event sink that binds to this category is the categorizer sink that resolves sender and recipients, and provides categorization information, such as the home server for each recipient, to the advanced queueing engine. In Exchange Server 2003, two event sinks are registered for the OnCategorize event: Exchange Categorizer and Outlook Mobile Access Push Categorizer (registered in the IIS metabase as Mobile Categorizer). Exchange Categorizer processes e-mail messages to resolve recipient information, expand distribution lists, check per-recipient restrictions, and much more. Mobile Categorizer implements up-to-date notifications for mobile devices.

  • SMTP Transport OnPostCategorize   This event occurs immediately after the message is categorized. At this point, all distribution lists (that have the local server as an expansion server) have been expanded, and the actual recipients are listed in the message transfer envelope. By default, no sink is registered for this event.

    Dica

    It is possible for Exchange Categorizer to split a message into multiple, separate copies, with different content or envelopes, during a process named bifurcation (explained later in this section). After categorization, the SMTP Transport OnPostCategorize event is triggered separately in an uncorrelated manner for each copy of the message. The message recipients might be distributed across several different message copies.

  • SMTP Transport OnGetMessageRouter   This event occurs when a message is intended for remote recipients and must be routed. This is not a single event, but an event category. There are multiple events that can be triggered for a sink that binds to this category. For example, the sink that determines the message type ID and next hop information in Exchange Server 2003 is named Exchange Router. The advanced queuing engine also uses the Exchange Router sink to update link state information, if the state of local messaging connectors changes, as explained in Message Routing Architecture.

    Dica

    The base SMTP service of Windows Server 2003 does not implement a router sink and sends messages point-to-point to the final recipient destination.

  • SMTP Transport OnMsgTrackLog   This event occurs when the advanced queuing engine processes a message, so that a sink can persist tracking information about the message in a log file. Exchange Server 2003 implements a MsgTrackLog sink for this event to write status information about all messages that pass through the advanced queuing engine to message tracking log files, stored in the \Program Files\Exchsrvr\<servername>.log directory (for example, \Program Files\Exchsrvr\Server01.log).

    Dica

    By default, message tracking is disabled.

  • SMTP Transport OnDnsResolveRecord   This event occurs when a domain name must be resolved to an IP address. Exchange Server 2003 registers a sink called Exchange LoadBalancer for this event, which is used to balance the load of outbound message transfer for routing group connectors with multiple bridgehead servers.

  • SMTP Transport OnMaxMsgSize   This event occurs when a submitted message contains content that exceeds the currently configured maximum message size. An event sink that handles this event can override the default blocking. By default, no sink is registered for this event.

  • SMTP StoreDriver   This event occurs when a message must be saved to disk or to the Exchange store. This is not a single event, but an event category. There are multiple events that can be triggered for a sink that binds to this category. For example, the advanced queuing engine triggers numerous StoreDriver events as a message passes through the transport subsystem. You can read more about StoreDriver event sinks later in this section.

    Dica

    SMTP StoreDriver events can be triggered by the advanced queuing engine or the protocol engine.

Message Queues in the Advanced Queuing Engine

The advanced queuing engine maintains a number of message queues in memory. A shared pool of threads services these internal queues. You can view these queues in Exchange System Manager. Specifically, the Exchange Queue Viewer snap-in, implemented in Exadmin.dll, communicates with the advanced queuing engine through the queue admin interface (PhatQAdm.dll) to list these queues and to perform queue management functions, such as freezing or unfreezing messages in a message queue. The following figure illustrates the most important message queues in the advanced queuing engine, as they relate to transport events.

377316fe-ba10-4e7d-8a0c-1dd82708eeca

The advanced queuing engine uses the following message queues:

  • Messages Pending Submission   This queue is also named the pre-submission queue. This is the entry point into the advanced queuing engine. The OnSubmission event is triggered for all messages in this queue. When this event succeeds, messages are placed into the pre-categorization queue.

  • Messages Awaiting Directory Lookup   This queue is also named the pre-categorization queue. It is a throttling queue for the categorizer. This queue contains messages before they reach the categorizer. The categorizer resolves the sender and recipient information against Active Directory, expands distribution lists, checks restrictions, applies per-sender and per-recipient limits, and more. The categorizer is discussed in more detail in the section, "Exchange Categorizer," in SMTP Transport Components.
    Messages are not in any queue during message categorization. A message that is being categorized is actually in the categorizer and is not listed in any queue. Thus, Queue Viewer might show an empty pre-categorization queue, while the Performance monitoring tool might show a positive count for the performance counter called Categorizations in progress. By default, the advanced queuing engine allows a maximum of 1,000 pending categorizations before retaining messages in the pre-categorization queue. This threshold can be changed using the following registry key.

    Location

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing

    Value

    MaxPendingCat

    Type

    REG_DWORD

    Value Data

    0x3E8

    Description

    This indicates the maximum number of pending categorizations before the advanced queuing engine starts retaining messages in the pre-categorization queue. The default is 1,000 connections.

  • Messages Waiting To Be Routed   This queue is also named the pre-routing queue. It holds all messages queued for local and remote delivery after they are categorized and post-categorization events are triggered. This is the point at which the advanced queuing engine decides which messages must go to the local delivery queue and which must be routed. For all messages to remote recipients, the advanced queuing engine calls the routing engine in an OnGetMessageRouter event to determine the next hop for each message in this queue and move the messages to their respective link queues.

  • Local Delivery   After categorization, messages pass through the pre-routing queue and enter the local delivery queue, if the recipient's mailbox resides on the local server running Exchange Server 2003. The message categorizer marks the recipient as local by setting a per-recipient property that indicates the destination server, based on the recipient's homeMDB attribute, which points to the recipient's mailbox store. For local recipients, the OnGetMessageRouter event is skipped and the advanced queuing engine moves messages to the local delivery queue before raising a StoreDriver event. The StoreDriver event informs the Exchange store driver that messages must be transferred to the Microsoft Exchange Information Store service.

  • Dynamic Delivery   These queues are domain queues with names that match the remote domain or next hop address on the link. The queue name identifies the destination.
    A special case is message transfer through Exchange MTA, which is often referred to as gateway delivery, because the MTA is also responsible for all MAPI-based connectors to non-Exchange messaging systems, as explained in more detail in X.400 Transport Architecture and Gateway Messaging Connectors Architecture. The SMTP transport subsystem uses the Exchange store driver to move messages to and from the MTA through the Exchange store. However, the advanced queuing engine does not know whether a message must be transferred through SMTP or the MTA until the routing engine returns this information. If the next hop of the recipient is over a non-SMTP connector (Routing Group Connectors are considered SMTP connectors unless the Routing Group Connector instance has an Exchange 5.5 bridgehead), the message is sent to the Exchange MTA delivery queue and then to the local delivery queue. From there the Exchange store driver sends it to the Exchange store. The recipient properties that the categorizer sets in the message transfer envelope are used to distinguish which recipients must be delivered to a local mailbox and which must be delivered to the Exchange MTA.

  • System   These queues contain messages with specific parameters. The following table lists the system queues that the advanced queuing engine maintains, in addition to delivery queues.

    System queues in the advanced queuing engine

    Queue Description

    Messages With an Unreachable Destination

    This queue contains messages that cannot reach their final destination server. For example, Exchange cannot determine a route or a connector to the final destination, or all available routes or connectors are labeled as down.

    Messages Queued for Deferred Delivery

    This queue contains messages that are queued for later delivery. This queue might contain messages that were sent by earlier versions of Microsoft Outlook, such as Microsoft Outlook 2000. Newer versions of Outlook store these types of messages in the Exchange store. Messages remain in the Messages Queued for Deferred Delivery queue until their scheduled delivery time.

    Dica

    This queue might also contain messages sent to a mailbox that was recently moved to another mailbox store or Exchange server, while waiting for directory replication to update the mailbox location.

    DSN Messages Pending Submission

    This queue contains delivery status notifications that are waiting to be rendered by Exchange. For example, if a message remains in a message queue for an extended period of time, the advanced queuing engine generates a delivery status notification to inform the sender that the message was delivered. Non-delivery reports (NDRs) are also delivery status notifications. For more information about delivery status notifications, see the following Microsoft Knowledge Base articles:

    Failed Message Retry

    This queue contains messages that failed a queue submission. Messages can fail a queue submission for several reasons, and the failure can occur before any other processing is done. If messages are corrupted or system resources are low, messages appear in this queue.

    Limiting the Number of Messages in Message Queues

    Each message in a message queue consumes at least four kilobytes (KB) of memory. Therefore, you might encounter insufficient memory with a very large queue. To avoid this situation, you can use the following registry parameter to limit the number of messages that are stored in the queue at a given time.

    Aviso

    Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.

    Location

    HKEY_LOCAL_MACHINE\Software\Microsoft\Exchange\Mailmsg

    Value

    MaxMessageObjects

    Type

    REG_DWORD

    Value Data

    0x000186a0

    Description

    If a value is not specified, the default is 0x000186a0 (100,000) messages. Specifying a lower value reduces the maximum number of messages that can reside in the advanced queuing engine, thus decreasing the maximum memory footprint for SMTP. After this limit is reached, each SMTP connection made to the server returns with an out-of-memory error. For example, if this value is reduced to 10,000 messages, SMTP refuses inbound messages in excess of 10,000 messages.