Internet Protocol Details
As mentioned, Exchange 2003 supports several Internet standards-based client protocols, including HTTP, POP3, IMAP4, and NNTP. These protocols are described in more detail in the following subsections.
HTTP
The Microsoft Exchange Information Store service includes native HTTP access to data. Every object in the Microsoft Exchange Information Store service is URL–accessible with short, easily understood names. Because every object in the Microsoft Exchange information store is URL–accessible, users have several different ways to access objects in mailboxes or public folder hierarchies. The URL for an object is based on its location in the hierarchy and usually contains the subject of the item.
When a user opens a message through Microsoft Outlook Web Access, the IIS request processor calls the Exchange HTTP ISAPI application that parses the information in the request and determines the following:
The action to be performed Exchange HTTP ISAPI determines whether the user is opening a mailbox, opening a folder, reading e-mail, creating e-mail, and so forth.
Browser information Exchange HTTP ISAPI determines the browser type, version, and rendering information.
The server then determines whether the user has permissions to access the item. If the user has access rights, the object state (read, unread), object type (folder, message, and others), and item type (message, appointment, contact) are determined.
The Exchange HTTP ISAPI extension then matches the object attribute to its corresponding form definition. If a form definition does not exist for a particular object attribute, the default form is used, (the one used to read an e-mail item). The Exchange HTTP ISAPI extension then parses the form and queries the information store to bind to the data. After receiving the data from the Microsoft Exchange Information Store service, the Exchange HTTP ISAPI extension renders the data in HTML or XML, based on the browser type and version, and the client displays the message.
The following steps show this process in more detail:
The browser sends a request for an e-mail message.
The browser issues a GET request for a URL, such as https://server/vroot/user/folder/message.eml. This URL does not have any query strings attached, which would be processed first, so the server returns a rendering of this resource based on its Message-Class and the default action configured for this class.
Exchange ISAPI processes the request.
When IIS receives the request, it is passed to the Exchange ISAPI component Davex.dll. This component parses the request for the following information and then sends the request to the Exchange store. The following table illustrates the passed item and its purpose.
The Microsoft Exchange Information store service then determines the item type.
The server verifies that the user has access to the item, determines the type of object (folder, message, task, and more), and returns the item type and its state (read, unread, and more) to the ISAPI application.
Exchange ISAPI selects the form.
The ISAPI program takes the object attributes and looks for a form definition in the forms registry that matches the object's type. If a matching form definition is not found, a default form stored in Wmtemplates.dll is used. If the browser language is not English, language specific strings are loaded from other template libraries in the \Exchsrvr\Res\Directory.
The Microsoft Exchange Information Store service retrieves data for the form.
After a form definition is found, the ISAPI program parses the form, calling the Microsoft Exchange Information Store service, to retrieve the data it references.
Exchange ISAPI renders the form.
When the data is returned from the Microsoft Exchange Information Store service, the form is rendered in the appropriate HTML and XML, and then goes to the client.
Davex.dll passed items and usage
| Passed item | Used to |
|---|---|
HTTP User-Agent Field header |
Determine the browser type, version, operating system, and how to render content |
HTTP Accept-Language header |
Determine the language for the rendered content |
HTTP Translate header |
Determine if the content should display in a browser or if it should return without rendering to a WebDAV application |
Query string |
Determine a specific action to perform |
WebDAV and XML
Web Distributed Authoring and Versioning (WebDAV) is an extension to the HTTP 1.1 protocol (RFC 2518). HTTP and WebDAV enable rich collaborative interaction with the information store in Exchange 2003. Exchange 2003 HTTP support enables adding, modifying, copying, moving, and searching of folders and items and manipulation of attributes on any object in the information store.
WebDAV creates improved performance and user experience over the basic Microsoft Outlook Web Access client by exploiting client-side data binding and rendering. For example, when you click the column header, you can sort the Inbox in several different ways, enabling views based on the sender's name, the message subject line, or received date. The browser caches the user interface elements, such as Internet Explorer HTML components, Microsoft Jscript libraries, XSL, and Graphics Interchange Format (GIF) files. When the user changes the sort criteria, the browser can reformat the user interface elements locally and query the server for the view data.
The following process shows how clients access items in their Inbox using WebDAV:
The client issues an HTTP GET request for the client's Inbox.
IIS receives the request on port 80 (unless you change this configuration) and sends the request to Davex.dll for processing using ExIPC.
The request is forwarded using ExIPC to the Exchange Store OLE DB driver, Exoledb.dll.
Exoledb.dll renders the request in a format that the Exchange store can process, sends the request to the Exchange store, and then retrieves the client's Inbox properties from the Exchange store.
After the clients Inbox properties are retrieved, Exchange 2003 routes the information back to the client using the same components that it used to process the client request.
POP3
Exchange Server 2003 implements a POP3 protocol stack that is compliant with RFC 1725, RFC 1734, and RFC 1939. Exchange 2003 supports the ten POP3 commands listed in the following table.
POP3 protocol command verbs
| Command | Description |
|---|---|
List |
Used to display the identifier number and the size (in bytes) of messages in the mailbox, or to display the number and size of a particular message. The list command uses the following syntax, where n is the message number that is returned by the list command: list or list n. |
Uidl |
Used to return a numeric listing of all messages in the mailbox and their associated unique IDs, or the unique ID for a particular message. The uidl command uses the following syntax, where n is the message number (as returned by the list command) of the uidl that you want to view: uidl or uidl n. |
Retr |
Used to retrieve a message from the server. You cannot use this command to retrieve a message that is marked as deleted. The retr command uses the following syntax, where n is the message number that is returned by the list command: retr n. |
Stat |
Returns the total number of messages in the mailbox and the total size (in bytes) of the messages. You cannot use this command to display more information about individual messages. To do this, you must use the list or retr commands, as appropriate. |
Dele |
Used to select a message for deletion. When you select a message for deletion, the message is deleted after you use the quit command to disconnect the client from the server. If the connection is interrupted unexpectedly, the messages are not deleted. The delete command uses the following syntax, where n is the message number that is returned by the list command: dele n. |
Rset |
Used to deselect all messages that are selected for deletion. |
Noop |
This translates to "no operation." Although this command does not perform any action, if the command is successful, the server replies with a positive response (OK+). You can use this command to test whether the server is online and receiving client requests. |
Top |
Used to display the message header and a particular number of lines of the message. Uses the following syntax, where x is the message number that you want to view, and y is the number of lines in the message that you want to display: top xy. |
Auth |
An IMAP command that is part of the POP3 specification, as detailed in Internet Engineering Task Force (IETF) Request for Comments (RFC) 1734. It permits you to use alternative IMAP4 authorization mechanisms. |
Quit |
Used to quit the current POP3 session and delete any messages that are selected for deletion. |
POP3 is considered a read only protocol. It only includes messages for requesting, reading, and deleting messages. To send messages, POP3 clients use the SMTP protocol.
The following steps illustrate the interprocess communication steps that ExIPC goes through when a client such as Microsoft Outlook accesses a mailbox on the Exchange server using the POP3 protocol.
IIS and Exchange Server shared memory architecture
.gif "e6620568-6c8c-4ddc-9585-b7cf67ef51d4")
The client logs on to the server and gives the command to check e-mail.
A Request Mail Message 1 command is created on the IIS side.
IIS allocates shared memory from the shared memory heap for the request. A corresponding handle is assigned to part of the shared memory. The handle, which functions as a placeholder or pointer to a referenced part of memory, is then placed in the circular memory queue (enqueued) in the direction of the Exchange information store.
On the Exchange store side, the ExIPC.DLL for POP3 checks for incoming POP3 requests. The DLL receives the Request Mail Message and removes the handle from the circular memory queue. The Exchange store-side POP3 stub references the handle to the data in the shared memory heap.
If there are no failures or performance problems on the Exchange store side, the ExIPC process is complete and the data is successfully communicated from the IIS to the Exchange store. If a queue is full or the Exchange store has stopped, an error message is returned.
A response (the mail message) is generated on the Exchange store side. The Exchange information store allocates the appropriate shared memory for the response from the shared memory heap. A corresponding handle is assigned to that shared memory. The handle is then enqueued in the direction of IIS.
IIS removes the handle from the circular queue, references the shared memory, and binds them together.
If there are no failures or performance problems on the IIS side, the response is complete and the data is successfully communicated from the Exchange store to IIS.
IMAP4
Exchange 2003 is IMAP4 rev 1 compliant, in accordance with RFC 2060, RFC 2088 and RFC 1731. IMAP is comprised of over 30 commands, through which messages can be searched, fetched, and expunged from the Exchange server. IMAP is well suited for online and offline use. IMAP can connect to multiple mailboxes (if permissions are in place) and public folders and can be used for non e-mail purposes, such as news services.
IMAP4 goes beyond the functionality available by using POP3. IMAP4 allows users to access any one of their folders, not just their Inbox. Because of this, it is more complex than POP3. However, it still adheres to the same standard of being a read-only protocol. Like POP3, IMAP4 also uses SMTP for sending e-mail.
Exchange 2003 supports the IMAP4 commands listed in the following table.
IMAP4 commands supported by Exchange Server 2003
| Command | Description |
|---|---|
APPEND |
Appends the literal argument as a new message to the end of the specified destination mailbox. This argument must be in the format of an RFC-822 message. |
AUTHENTICATE |
Indicates an authentication mechanism to the server (for example, AUTHENTICATE KERBEROS_V5). |
CAPABILITY |
Used to request a list of capabilities that the server supports. |
CHECK |
Used to request a checkpoint of the currently selected mailbox. A checkpoint refers to any implementation-dependent details associated with the mailbox (for example, resolving the server's in-memory state of the mailbox with the state on its disk) that is not executed as part of each command. |
CLOSE |
Permanently removes from the currently selected mailbox all messages that have the \Deleted flag set, and returns to authenticated state from selected state. |
COPY |
Used to copy the specified message(s) to the end of the specified destination mailbox. The flags and internal date of the message(s) are preserved in the copy. |
CREATE |
Used to create a mailbox with the particular name. An OK response is returned only if a new mailbox with that name is created. |
DELETE |
Permanently removes the mailbox with the particular name. A tagged OK response is returned only if the mailbox is deleted. |
EXAMINE |
The same as SELECT and returns the same output. However, the selected mailbox is identified as read-only. No changes to the permanent state of the mailbox, including per-user state, are permitted. |
EXPUNGE |
Permanently removes from the currently selected mailbox all messages that have the \Deleted flag set. |
FETCH |
Retrieves data associated with a message in the mailbox. |
LIST |
Returns a subset of names from the complete set of all names available to the client. |
LOGIN |
Identifies the client to the server and carries the plain text password authenticating this user. |
LOGOUT |
Informs the server that the client is done with the connection. |
LSUB |
Returns a subset of names from the set of names that the user declared as being "active" or "subscribed." |
NOOP |
This translates to "no operation." Although this command does not perform any action, if the command is successful, the server replies with a positive response (OK+). You can use this command to test whether the server is online and receiving client requests. |
RENAME |
Changes the name of a mailbox. A tagged OK response is returned only if the mailbox is renamed. |
SEARCH |
Searches the mailbox for messages that match the specified searching criteria. Searching criteria consist of one or more search keys. |
SELECT |
Selects a mailbox so messages in the mailbox can be accessed. |
STATUS |
Requests the status of the indicated mailbox. It does not change the currently selected mailbox, nor does it affect the state of any messages in the queried mailbox. |
STORE |
Alters data associated with a message in the mailbox. |
SUBSCRIBE |
Adds the specified mailbox name to the server's set of "active" or "subscribed" mailboxes as returned by the LSUB command. This command returns a tagged OK response only if the subscription is successful. |
UID |
This command has two forms. In the first form, it takes as its arguments a COPY, FETCH, or STORE command with arguments appropriate for the associated command. In the second form, the UID command takes a SEARCH command with SEARCH command arguments. |
UNSUBSCRIBE |
Removes the specified mailbox name from the server's set of "active" or "subscribed" mailboxes, as returned by the LSUB command. This command returns a tagged OK response only if un-subscription is successful. |
NNTP
Network News Transfer Protocol (NNTP) is a TCP/IP protocol based on text strings that are sent bi-directionally over seven-bit ASCII TCP channels. The IETF owns the NNTP protocol, which is defined in RFC 977. NNTP is commonly referred to as the Internet News Protocol, because it contains the rules for transporting news articles from one computer to another. NNTP is discussed here as a client/server protocol. It also encompasses server-to-server based news transfer.
The NNTP service in Windows is designed to support a stand-alone newsgroup server that can easily create group discussions. When Exchange 2003 is installed, the NNTP service is enhanced with the ability to interface with other news servers through news feeds. The NNTP service can communicate with external NNTP servers to make popular USENET groups available to users.
The standard storage location for the NNTP service is in one or more directories in the file system. With Exchange Server 2003, the NNTP service can also store newsgroups in the public folders of any available public folder tree. Internet Newsgroups folder is the default location for newsgroups. The NNTP service uses virtual directories to reference these locations.
You can arrange multiple news servers in a master server/subordinate server layout. This enables clients to connect to a large group of servers and still maintain accurate views of newsgroup content. A bank or group of servers provides additional scalability for many clients and provides fault tolerance if a subordinate server goes offline.
The Exchange Server 2003 implementation of NNTP provides the following additional features for this protocol:
Content indexing provides search features for public folders.
Full news feeds are accepted independent of back-end storage.
MAPI or NNTP clients can read or post to newsgroups that are supported by the Exchange information store.
Configuration Settings in Active Directory
Although Exchange integrates with IIS, as soon as Exchange 2003 is installed, protocol virtual servers are managed by Exchange System Manager, and not by Internet Services Manager. When you add, remove, or configure an item in Exchange System Manager, the configuration changes are first saved to the Microsoft Active Directory directory service and then replicated to the IIS metabase, on the appropriate Exchange 2003 server, by the Directory Service/Metabase Synchronization (DS2MB) function that runs in the System Attendant process.
Note
You can view a semi-graphical representation of Exchange 2003 configuration information stored in Active Directory in Microsoft Knowledge Base article 252370, "Layout of Exchange Configuration in Active Directory."
Configuration Settings in the Metabase
The IIS metabase is a hierarchical database that is used to store configuration values for IIS and Exchange 2003. The IIS metabase is both a storage mechanism and an application programming interface (API) set used to make changes to the configuration parameters.
The function of the DS2MB process is to transfer configuration information from Active Directory to the Exchange server's local IIS metabase. For performance and scalability reasons, this configuration is stored in the local IIS metabase instead of in the registry.
Note
On computers running Windows 2000 Server, the IIS metabase is located at System32\Inetsrv\Metabase.bin. On computers running Windows Server 2003, the IIS metabase is located at metabase.xml. The IIS metabase can be manipulated through a variety of tools. On computers running Windows Server 2003, the built-in IISCNFG tool can be used. On computers running Windows 2000 Server, the MetaEdit 2.2 or later tool from the IIS Resource Kit is a good option. You can download the IIS 6.0 Resource Kit from the Internet Information Services (IIS) 6.0 Resource Kit Tools Web site.
Paths in the metabase are named keys. Properties can be set at each key, and each property can have attributes that customize that property. All identifiers that are present in the directory service image of the subtree are required in the metabase, including identifiers such as KeyType. Additionally, the Relative Distinguished Name of the object in the directory is mapped directly to the key name in the metabase.
IIS Metabase Updates Through DS2MB
DS2MB is a subprocess that is launched when System Attendant is started, and every 15 minutes thereafter. DS2MB copies all subtrees from Active Directory without changing the shape of the subtree. This is a one-way write from Active Directory to the metabase; the metabase never writes to Active Directory. It does not add or compute any attribute when copying.
Note
The DS2MB process overwrites changes that are made to Exchange virtual servers and directories using the IIS snap-in with information that is contained in Active Directory.
The operation of SMTP, POP3, IMAP4, and HTTP depends on the replication by DS2MB. Not all settings are synchronized from Active Directory, some are written to the metabase directly during the installation of Exchange.
Upon instantiation, DS2MB registers with the configuration domain controller. The configuration domain controller notifies DS2MB, within 15 seconds, of any changes that are made to the Exchange configuration . As soon as the change is replicated to the configuration domain controller, it must be replicated to the metabase by DS2MB.
High Water Marks
High water marks are entries in the metabase that enable DS2MB to track changes that have been synchronized from Active Directory. High water mark entries are entered in the IIS metabase in the form of GUIDs. These GUIDs appear under the [/DS2MB/HighWaterMarks] node in the metabase, as illustrated below:
[/DS2MB/HighWaterMarks]
KeyType : (STRING) "Ds2mbHighwatermarks"
[/DS2MB/HighWaterMarks/{BE583A06-9083-400F-954C-CF4ACCA78B04}]
[/DS2MB/HighWaterMarks/{028C8F78-8CF0-43D9-9B35-9819D538849F}]
[/DS2MB/HighWaterMarks/{84ECD394-05BB-4661-BA1D-81D3E32BF804}]
Because DS2MB handles the entry and synchronization of high water marks in the metabase, there is usually no reason to adjust or manage this information. However, there are known scenarios in which the resolution includes deleting the high water mark entries from the metabase to reset them.
Front-End Server Architecture
A front-end server is a server running Exchange Server 2003 that does not host a database (except when also serving as an SMTP server), but instead forwards client requests to the back-end server for processing. The front-end server uses Lightweight Directory Access Protocol (LDAP) to query Active Directory to determine which back-end server hosts the user's mailbox. A back-end server is a server running Exchange Server 2003 that maintains at least one database.
This architecture is available only for RPC over HTTP, HTTP/WebDAV, POP3, and IMAP4 clients. It is not intended for MAPI or NNTP clients. Clients that are supported connect to a front-end server that proxies client commands to the user's back-end server, which hosts an Exchange information store.
This functional division between a front-end server and a back-end server provides several benefits. For example:
Single Namespace As multiple back-end servers are configured to handle additional mailboxes, it is best to identify all the servers with a single name. You can refer to a front-end server with a single name, and it can proxy user requests to the correct back-end server containing that user's mailbox. If multiple front-end servers are configured to manage a high number of requests, a single namespace for these servers is maintained by configuring the Domain Name System (DNS) with one name mapped to the IP address on the server. It is not important which front-end server the client connects to.
Offload SSL Encrypting and decrypting message traffic uses many CPU cycles. A front-end server can perform encryption work, giving the back-end server more cycles to manage the mailbox and public folder information stores.
Public Folder Referrals for IMAP4 Clients Many IMAP4 clients do not support referrals. With this architecture, the front-end server can retrieve public folders that exist on a server other than the user's e-mail server.
Server Location You can put the back-end servers that contain the databases behind a firewall for increased protection. You can configure the firewall to allow traffic only from the front-end server. Additionally, you can put a reverse proxy (such as ISA Server) in front of a front-end server and only publish the front-end server. It is not necessary to publish the back-end mailbox servers to the Internet. Therefore, you can configure your firewalls and reverse proxies to allow traffic only to the front-end server.
Considerations When Using Front-End Servers
You can configure either Exchange Server 2003 Standard Edition or Exchange Server 2003 Enterprise Edition for use as a front-end server in a front-end and back-end server configuration. The following considerations apply when you configure either edition as a front-end server:
If the front-end server accepts SMTP mail from the Internet, you must start the Microsoft Exchange Information Store service and mount at least one mailbox store. In certain situations (most notably in the generation of non delivery reports), the SMTP service requires a mailbox store to perform a conversion.
If the mailbox store is not mounted, messages that must be converted are stuck in the local delivery queue. For security reasons, make sure that user mailboxes are not homed on the mailbox store of a front-end server. If there are servers that are running Exchange Server 5.5 in the same site (routing group), you must configure the Microsoft Exchange MTA Stacks service to run on the front-end server. In this configuration, the MTAs can bind and transfer mail by using RPCs.
If X.400 connectors or Exchange Development Kit (EDK) gateway connectors are homed on the front-end server, the MTA service must also run on the front-end server. If you delete all public folder and mailbox stores, you cannot change the configuration by using Internet Services Manager.
If you must change the configuration by using Internet Services Manager, for example when you change an SSL encryption configuration, make sure that you either complete the procedures that this guide describes before you remove the stores, or leave the private information store intact on the front-end server.
When you create a front-end server, do not delete the First Storage Group object in Exchange System Manager. The Microsoft Exchange Information Store service (and its related services) depends on the First Storage Group object.