Communications

Exchange Edge Transport Servers at Microsoft

Kay Unkroth

At a Glance:

• The Exchange Edge Transport server role
• Setting up a test lab
• Transport agents and events
• Agent internals

Download the code for this article: ExchangeEdge2007_10.exe (354KB)

Microsoft receives approximately 13 million message submission attempts from the Internet on an average business day—and blocks over 10.5 million of these as not legitimate. In critical

situations, such as during spam attacks or virus outbreaks on the Internet, the volume can swell to more than 90 million. Of course, this is a Microsoft-specific finding, yet scamming, spamming, phishing, e-mail-borne viruses, directory harvesting, distributed denial of service (DDoS) attacks, and similar concerns are not specific to Microsoft. Facing such issues, how do you ensure delivery of all legitimate messages to your users while keeping the flood of illegitimate, malicious content away from your messaging environment?

One way to achieve reliable messaging protection is to deploy Exchange Server 2007 Edge Transport servers and Forefront Security for Exchange Server in a perimeter network between the Internet and your production environment. This places an organized team of more than 10 Edge Transport agents at your perimeter to help protect your production systems and users. The value of blocking undesired content at the earliest point possible is obvious. Ideally, this occurs even before the content is delivered to your servers, considering the load your messaging environment might otherwise have to sustain in a critical situation. Yet there is a lot more to messaging protection than simply blocking messages.

This is the first of a two-part series that discusses the architecture and key features of anti-spam and antivirus agents available with Exchange Server 2007 and Forefront Security for Exchange Server. To keep the explanations realistic and practical, I will show you in this first article how to build a test lab that mirrors the messaging protection design Microsoft uses in its own corporate production environment. Then I'll cover the Edge Transport architecture of Exchange Server 2007 in more detail.

Throughout this article, I will make heavy use of scripts and batch files to automate the most important configuration tasks. These files include comments that explain the individual steps performed. You can find the scripts in the download available from the TechNet Magazine Web site at technetmagazine.com/code07.aspx.

Edge Transport Topology

Figure 1 illustrates the Edge Transport design that Microsoft IT implemented in the corporate production environment. There are some very interesting design features I'd like to point out before taking you through the Edge Transport architecture. If you are interested in the full implementation details, see the IT Showcase white paper referenced in the "Exchange Resources" sidebar.

Figure 1** Edge Transport topology at Microsoft **(Click the image for a larger view)

As you analyze Figure 1, notice that the Edge Transport topology at Microsoft is fully redundant. There is no single point of failure in any location. For load balancing, Microsoft IT uses DNS round robin externally and messaging connectors with multiple bridgeheads internally. Also noteworthy is the fact that all transport servers (Hub Transport and Edge Transport) run Forefront Security for Exchange Server for virus scanning. This enables Microsoft IT to scan all inbound, outbound, and internal messages as soon as the messages reach a transport server in the message routing topology.

Another important security-related design feature is that the Edge Transport servers are not part of the corporate Active Directory® environment. In fact, it is not necessary to deploy Edge Transport servers in any Active Directory forest. However, to maintain a consistent management framework, apply a common set of policies, and support single sign-on, Microsoft IT deploys all Edge Transport servers in an extranet forest, separate from the production environment.

Finally, you can clearly see that all messages from the Internet reach Microsoft through Edge Transport servers that are stationed in North America. The Edge Transport servers based in Dublin and Singapore only handle outbound messages. The advantage of concentrating all inbound traffic onto the Edge Transport servers in North America is to centralize security and anti-spam controls while avoiding the transfer of outbound messages from large regional datacenters across the internal messaging backbone.

Microsoft IT Systems Engineer Omesh Desai designed the Edge Transport topology at Microsoft. When I asked Omesh about the most important design features, he said, "In our Edge Transport design, we capitalize on native Exchange anti-spam and antivirus capabilities for messaging protection at multiple layers in the messaging backbone. Messaging perimeter security is first priority but a simple administration and management model is also important for us. Edge Transport servers help us increase security through tighter firewall configurations while increasing the accuracy of spam filtering through IP reputation services, automatic content filter updates, safelist aggregation, and e-mail postmark validation. All internal server-to-server communication is encrypted by default and we also encrypt communication with external destinations if possible.

"We use two dual-core 64-bit processors and eight gigabytes of memory in our Edge Transport servers. With six of these servers, load-balanced across two data centers, we have sufficient capacities to sustain even large spikes in message submissions, such as during virus outbreaks on the Internet."

Edge Transport Test Lab

Test Lab Setup

Because it is a best practice to use non-existing domain names and private IP addresses, I use AdventureWorks.com and IP addresses from 192.168.xxx.0-24 ranges. There are no redundant systems or firewall arrays because I'm not testing load balancing or fault tolerance. (And, of course, the actual firewall systems that Microsoft IT uses can't be discussed for security reasons.) Such details are not important for this test environment anyway.

My test environment uses ISA Server 2006 because it is likely one of the most common firewall systems used with Exchange Server 2007. Running ISA Server 2006 on both the outer and the inner firewall helps to keep the complexity of the test environment at a moderate level, yet for production environments I recommend using diverse outer and inner firewall systems to increase security. I did not deploy IP Security (IPsec) policies or prepare the environment for Transport Layer Security (TLS) as these topics are outside the scope of this article.

However, I did use virtual machines and 32-bit evaluation software, which you can download from the Microsoft Web site. Microsoft does not support the 32-bit version of Exchange Server 2007 in production, but that's not an issue in a test environment.

My test lab relies on default settings wherever possible. Only the IP configuration, firewalls, and DNS zones require special attention prior to running Exchange Server 2007 setup and subscribing the Edge Transport server to the production environment. For details regarding the IP configuration, check out the Test Lab—IP Configuration.xls file you'll find in the companion download. If you use the same IP address assignments, you can quickly configure the outer firewall, called ISA01, by running the ISA01_Firewall_Policies.vbs script directly on the ISA01 computer, and use ISA02_Firewall_Policies.vbs for the inner firewall (ISA02). The companion download also includes batch files to configure the DNS servers (INTERNET01_DNS_Config.bat, AD01_DNS_Config.bat, and AD02_DNS_Config.bat). Because these batch files use the DNS command-line tool (dnscmd.exe), you need to have Windows Support Tools installed; otherwise you have to create the DNS records manually using the DNS console.

To avoid any interference from existing environments, my test lab is not connected to the Internet. This isolation is a good precaution. It causes all downloads of signature updates, IP reputation, and content filter updates to fail, but this is not critical for testing purposes. To avoid error messages, go to Scanner Updates in the Forefront Server Security Administrator console, then set the update frequency for all scanning engines to once and specify a date in the past.

To explore these features in action, it's good practice to build a test environment—common sense suggests never using a production system for testing purposes. At the minimum, it takes one server running Exchange Server 2007 for the Mailbox, Client Access, and Hub Transport server roles. You'll need a second Exchange server for the Edge Transport server role. You could skip the Edge Transport server installation if you deploy all transport agents on the multiple-role server by running the Install-AntispamAgents.ps1 script (you can find it on Hub Transport servers in the %ProgramFiles%\Microsoft\Exchange Server\Scripts folder). But this approach would hardly resemble the Microsoft IT deployment. For a realistic test lab, you'll need to include a few more servers. Figure 2 shows the test environment I used for researching this article. A more detailed illustration is available in the companion download. For more information about setting up the lab, see the "Test Lab Setup" sidebar.

Figure 2** Edge Transport test environment **(Click the image for a larger view)

During subscription of the Edge Transport server and configuration of associated connectors, Microsoft IT removes all default connectors and then proceeds to create four send connectors to communicate efficiently with different types of Simple Mail Transfer Protocol (SMTP) hosts and internal Hub Transport servers. The first send connector is a general Internet connector for all of the destinations that do not match specific address space definitions.

The second send connector is an Internet connector with detailed address space definitions for known destinations that do not support Extended SMTP (HELO domains). By setting the ForceHELO parameter for this connector to $true, Microsoft IT avoids an unnecessary sequence of EHLO, Failure Response 500, HELO when establishing SMTP connections.

The third send connector is an Internet connector with detailed address space definitions for partners and other remote domains that support TLS to communicate securely over encrypted connections (TLS domains). This connector has the RequireTLS parameter set to $true.

The fourth send connector is an inbound connector to transfer received Internet messages to Hub Transport servers in the corporate environment. Again, for more details regarding the Edge Transport server configuration, see the IT Showcase white paper referenced in the "Exchange Resources" sidebar at the end of this article.

To apply a Microsoft IT-style connector topology to the test lab, I relied on a procedure based on scripts that Omesh created for internal Microsoft IT use. For security reasons, I changed and shortened the individual commands drastically, but the resulting connector topology still corresponds to the Microsoft IT topology. Here are the steps:

1. On the multiple-role Exchange server (HUB-MBX-01) and the Edge Transport server (EDGE01), remove the default connectors.
2. On HUB-MBX-01, create a new receive connector by running the HUB-MBX-01_recv_connector.ps1 script that you can find in the download for this article.
3. On EDGE01, create two new receive connectors for internal and external messaging connectivity by running the EDGE01_recv_connector.ps1 script.
4. On EDGE01, create a subscription file by running this command:

    New-EdgeSubscription -FileName 
    "c:\subscriptionfile.xml" 

Then copy the resulting subscription file to the Hub Transport server's root folder (c:\subscriptionfile.xml).

5. On HUB-MBX-01, make sure the path to the subscription file is c:\subscriptionfile.xml and then run the HUB-MBX-01_complete_subscription.ps1 script. This script imports the subscription file for Edge Synchronization without automatic send connector creation, creates the send connectors for Internet and internal connectivity, and replicates the resulting configuration to the Edge Transport server via Edge Synchronization.

6. Verify the configuration by sending test messages as Contoso.User@contoso.com and Fabrikam.User@fabrikam.com from the Internet host to Administrator@adventureworks.com and replying to the received messages to ensure that inbound and outbound message transfer works.

I suggest you open the individual script files in Notepad and analyze the cmdlets and parameters these scripts use to perform the configuration. Detailed information about these cmdlets and parameters is available online in the product documentation for Exchange Server 2007.

Edge Transport Architecture

Now let's get in touch with the Edge Transport agents, which have been waiting for somebody to say HELO or EHLO since the moment I installed the Edge Transport server role. If you run the Get-TransportAgent cmdlet on the Edge Transport server, you should see the 11 entries that are listed in Figure 3. All agents are enabled by default to provide messaging protection with appropriate settings.

Figure 3 Transport agents

The Get-TransportAgent cmdlet and Figure 3 list the agents in the order of their priority, yet this is not the order in which the agents perform their work. The work order depends primarily on the sequence of SMTP receive events and routing events for which the agents are registered. To see how agents and events fit together, have a look at the diagram shown in Figure 4, which illustrates how transport agents integrate into the Edge Transport architecture.

Figure 4** Transport agents within the Edge Transport architecture **(Click the image for a larger view)

Transport events occur at various stages during message processing to invoke additional code for spam filtering, virus scanning, and other tasks. In this loosely coupled and extensible design, the Edge Transport process (EdgeTransport.exe) assumes the role of the event source. The event handlers, in other words the transport agents, are managed delegate objects based on the Microsoft® .NET Framework 2.0, registered with the event source to receive callback notifications.

Figure 5 shows the event registrations for all agents installed on an Edge Transport server with Forefront Security. These registrations are perhaps somewhat difficult to sort out due to the large number of agent registrations, but don't despair. If you run the Get-TransportPipeline | Format-List command on an Edge Transport server, you can analyze the registrations for each individual transport event more conveniently. Just make sure that the Microsoft Exchange Transport service (MSExchangeTransport.exe) is running and that you have sent at least one message through the Edge Transport server since the last service restart. As the output reveals, multiple agents can register for the same event type and individual agents can register for multiple events. The event registrations just depend on the processing requirements of the corresponding agent.

Figure 5** Event registrations for transport agents **(Click the image for a larger view)

One of the most important events is the OnSubmittedMessage routing event, triggered when a message reaches the submission queue. All messages must pass through this queue whether they come in through SMTP, the file system, or any other mechanism. The categorizer is a core component of the Exchange Server transport architecture, responsible for recipient resolution, message bifurcation and routing, and delivery status notification (DSN) generation. The OnSubmittedMessage event is therefore a perfect registration choice for agents that must process all received messages. The FSE Routing Agent is a Forefront Security component registered for the OnSubmittedMessage event in order to pass all received messages to the virus-scanning engines. Because the FSE Routing Agent is registered for the OnSubmittedMessage event, no message can bypass the antivirus solution.

So why don't all agents just register for the OnSubmittedMessage event and consider the job done? Because you want to block unwanted messages at the earliest point possible, before your server confirms a successful delivery. Otherwise, your servers might have to process 90 million unwanted messages during a spam or virus attack, possibly having to generate 90 million non-delivery reports (NDRs), which in turn could pose a serious threat to innocent bystanders. Spam and virus attacks almost always use falsified originator information. Sending millions of NDRs to recipients that did not create the original messages is not only a waste of your resources and the resources of the target organizations, it is also an opportunity for malicious users to launch mail-flooding and DDoS attacks. It is important to stop malicious senders in their tracks for your own protection and the protection of others.

To block a message efficiently, a transport agent must interrupt the SMTP conversation with the remote host before your server confirms the receipt of the data with a 250 OK status code. According to the SMTP store-and-forward principle, your server can safely discard any received data without generating NDRs if message delivery was not confirmed. SMTP receive agents can accomplish this. They interact with the SMTP session because the transport pipeline invokes these agents based on SMTP receive events as the remote host connects to the server, establishes an SMTP session, transmits SMTP verbs, submits messages, and terminates the connection. (The SMTP receive events related to each step are listed in Figure 6.) Because of the ability to reject messages prior to delivery and disconnect remote SMTP hosts, all Exchange Server 2007 anti-spam agents are implemented as SMTP receive agents.

Figure 6 SMTP Session events

It is important to recognize the difference between SMTP receive agents and routing agents with regard to their processing context. While routing agents have full access to the message properties, SMTP receive agents are more context-sensitive because these agents interact with the SMTP session. For example, it is not possible for a spam filter to act on message properties until the remote host has actually transferred the message. Thus it is important to register the agent for the correct SMTP receive event. See the "Transport Agent Development" sidebar for a deeper look.

Exchange Resources

• Microsoft Exchange Server 2007 Edge Transport and Messaging Protection (Microsoft IT Showcase Technical White Paper)
• Windows PowerShell Script Center
• Exchange Server 2007 TechCenter
• Microsoft Exchange Team Blog "You Had Me From EHLO"

Stay Tuned

Let's take a break before going deeper into the transport architecture and test scenarios. I covered a lot of ground, ranging from a Microsoft IT-style deployment of Edge Transport servers to the inner events triggered in the transport pipeline during message processing. In the next installment of this two-part series, I'll continue by analyzing the behavior of Edge Transport agents in a few interesting test scenarios.

In the meantime, I recommend downloading the 90-day trial version of Microsoft Visual Studio 2005 Professional Edition (go.microsoft.com/fwlink/?LinkId=98043) and follow Steve's explanations to compile and install sample agents in your test environment. Even if you are not a developer, you will find that accomplishing these tasks is very straightforward. It would not surprise me to see an increasing number of business applications rely on custom agents, given how convenient it is in Exchange Server 2007 to develop these components with Visual Studio 2005.

Kay Unkroth is an entrepreneur who has worked as a support engineer, system developer, consultant, trainer, and author with focus on Microsoft server technologies for more than 15 years. Kay is also co-founder and chairman of Biblioso Corporation, a company that specializes in managed documentation and localization services.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.