Chapter 13: Configuring the Firewall as a Filtering SMTP Relay

One of the optional components included with the ISA Server 2004 is the SMTP Message Screener. The SMTP Message Screener can inspect SMTP messages at the application layer relay or reject messages based on parameters you configure. The SMTP Message Screener can evaluate incoming SMTP mail based on the following characteristics:

• Sender mail account and sender domain name
• Attachments name, attachment extension and attachment size
• Keywords included in the subject line and body of text/plain and text/html messages

For example, a common attachment extension for Internet worms is the .pif extension. Because very few or no legitimate e-mail messages contain attachments with the .pif extension, you can configure the filter to match messages with attachments with this extension and perform one of the following actions:

• Delete the message
• Hold the message
• Forward the message to a specified e-mail account

The SMTP Message Screener is an integral part of your e-mail defense in-depth scheme. Internet worms and viruses, in addition to spam, represent some of the most significant risks to your network. Worms and viruses can attack network servers, services and workstations throughout the Internal network. Spam clogs Internal network bandwidth and consumes employee time, costing many thousands, even millions, of dollars per month in employee productivity.

E-mail defense in depth allows you to distribute the processing of incoming and outgoing e-mail messages. SMTP message evaluation is a processor-intensive activity, and the more machines the load is distributed to, the more efficient the process. You can use the ISA Server 2004 SMTP Message Screener together with the Exchange SMTP Gateway Server to provide an ideal level of e-mail defense in depth.

In the example discussed in this document, we will configure the ISA Server 2004 firewall as an inbound and outbound SMTP relay. The inbound SMTP relay component will accept incoming mail from external SMTP servers destined for e-mail domains that you manage on your Exchange Server. The outbound SMTP relay is used to screen e-mail send out from the Exchange Server to e-mail domains on the Internet (e-mail domains that you do not host or control).

To achieve these goals, you will perform the following steps:

• Restore the system to its post-installation state
• Assign a second IP address to the Internal interface of the ISA Server 2004 firewall
• Install and configure the SMTP Service
• Install the SMTP Message Screener
• Create the SMTP Server Publishing Rules
• Configure SMTP Message Screener logging
• Test SMTP Filtering

Restore the System to its Post-installation State

To fully test the inbound and outbound SMTP relay configuration in this scenario, we will return the machine to its post-installation state so that other Access Rules do not interfere with the scenario development. In a production environment, you would leave your current Access Rules intact and add the Server Publishing Rules required to create the inbound and outbound SMTP relays.

Perform the following steps to restore the ISA Server 2004 firewall machine to its post-installation state:

1. Open the Microsoft Internet Security and Acceleration Server 2004 management console and right-click the server name. Click the Restore command.
2. In the Restore Configuration dialog box, select the backup file you created earlier and click Restore.
3. In the Type Password to Open File dialog box, enter the password you assigned to the file in the Password text box and click OK.
4. Click OK in the Importing dialog box after you see the message The configuration was successfully restored.
5. Click Apply to save the changes and update the firewall policy.
6. Select Save the changes and restart the service(s) in the ISA Server Warning dialog box, and click OK.
7. Click OK in the Apply New Configuration dialog box.

Assign a second IP address to the Internal interface of the ISA Server 2004 firewall

We will add a second IP address to the Internal interface of the ISA Server 2004 firewall machine. This will allow us to publish the outbound SMTP relay on a different IP address than the inbound SMTP relay. While this is not required, it greatly simplifies tracking which relay is to be used by particular clients.

Perform the following steps to add a second IP address to the Internal interface of the ISA Server 2004 firewall machine:

1. At the ISA Server 2004 firewall machine, right-click My Network Places on the desktop and click Properties.
2. In the Network Connections window, right-click the LAN interface and click Properties.
3. In the LAN Properties dialog box, scroll through the This connection uses the following items list and double-click Internet Protocol (TCP/IP).
4. In the Internet Protocol (TCP/IP) Properties dialog box, click the Advanced button.
5. In the Advanced TCP/IP Settings dialog box, click the IP Settings tab. In the IP addresses frame, click Add.
6. In the TCP/IP Address dialog box, enter 10.0.0.10 in the IP address text box. Enter 255.255.255.0 in the Subnet mask text box. Click Add.
   
7. The IP address 10.0.0.10 now appears second in the list of IP addresses. Click OK.
8. Click OK in the Internet Protocol (TCP/IP) Properties dialog box.
9. Click OK in the LAN Properties dialog box.

Install and Configure the SMTP Service

Install the IIS 6.0 SMTP service before the ISA Server 2004 SMTP Message Screener. The SMTP service works together with the SMTP Message Screener to examine and block offending e-mail messages.

Perform the following steps to install the IIS 6.0 SMTP service:

1. Click Start and point to Control Panel. Click Add or Remove Programs.
2. In the Add or Remove Programs window, click Add/Remove Window Components on the left side of the window.
3. On the Windows Components page, click Application Server in the list of Components, and click Details.
4. In the Application Server dialog box, click Internet Information Services (IIS), and click Details.
5. In the Internet Information Services (IIS) dialog box, place a check mark in the SMTP Service check box and click OK.
   
6. Click OK in the Application Server dialog box.
7. Click Next on the Windows Components page.
8. Click OK in the Insert Disk dialog box.
9. Enter the path to the i386 folder in the Copy file from text box in the Files Needed dialog box.
10. Click Finish in the Completing the Windows Components Wizard page.

The next step is to configure the SMTP server service to support inbound and outbound relay:

1. Click Start and point to Administrative Tools. Click Internet Information Services (IIS) Manager.
2. In the Internet Information Services (IIS) Manager console, expand the computer name in the left pane of the console. Right-click the Default SMTP Virtual Server and click Properties.
3. In the Default SMTP Virtual Server Properties dialog box, click the Access tab.
4. On the Access tab, click the Relay button in the Relay restrictions frame.
5. In the Relay Restrictions dialog box, confirm that the Only the list below option is selected. Then click Add.
6. In the Computer dialog box, select the Single computer option and enter the IP address of the Exchange Server in the IP address text box. In this example the IP address of the Exchange Server is 10.0.0.2. Click OK.
   
7. Click OK in the Relay Restrictions dialog box.
8. Click Apply and OK in the Default SMTP Virtual Server Properties dialog box.
9. Expand the Default SMTP Virtual Server node in the left pane of the console and right-click the Domains node. Point to New and click Domain.
10. On the Welcome to the New SMTP Domain Wizard page, select Remote and click Next.
11. On the Domain Name page, enter the domain hosted on the Internal network in the Name text box. This is the domain that you want the SMTP relay on the ISA Server 2004 firewall to accept incoming mail from Internet SMTP servers. In this example, the Internal network domain is msfirewall.org, so enter that. Click Finish.
12. Double-click the msfirewall.org domain in the right pane of the console.
13. In the msfirewall.org Properties dialog box, place a check mark in the Allow incoming mail to be relayed to this domain check box. Select Forward all mail to smart host. Enter the IP address of the Exchange Server on the Internal network in the text box, enclosed in straight brackets. In our current example, the IP address of the Exchange Server on the Internal network is 10.0.0.2, so we will enter [10.0.0.2]. Click Apply and OK.
    
14. Right-click the Default SMTP Virtual Server node and click Stop. Right-click the Default SMTP Virtual Server node and click Start.

Install the SMTP Message Screener

The SMTP Message Screener is an optional ISA Server 2004 component. This feature integrates with the IIS 6.0 SMTP service to examine and block SMTP mail based on parameters you configure in the Message Screener.

Perform the following steps to install the SMTP Message Screener on the ISA Server 2004 firewall computer:

1. Close the Microsoft Internet Security and Acceleration Server 2004 management console.
2. Locate the ISA Server 2004 installation media and double-click the isaautorun.exe file.
3. In the autorun menu, click the Install ISA Server 2004 icon.
4. Click Next on the Welcome to the Installation Wizard for Microsoft ISA Server 2004 page.
5. On the Program Maintenance page, select Modify and click Next.
   
6. On the Custom Setup page, click the Message Screener option and This feature, and all subfeatures, will be installed on local hard drive. Click Next.
   
7. Click Install on the Ready to Modify the Program page.
8. Put a check mark in the Invoke ISA Server Management when the wizard closes check box and click Finish on the Installation Wizard Completed page.
9. Close the Autorun menu.

Create the SMTP Server Publishing Rules

The SMTP Message Screener works together with SMTP Server Publishing Rules. Each SMTP Server Publishing Rule can be configured with a custom set of SMTP Message Screener parameters. This allows you to create different e-mail screening policies for the inbound and outbound SMTP relays. Different SMTP Message Screener configurations allow you to block different e-mail messages coming into the network versus what gets blocked on the way out.

Perform the following steps to create the Server Publishing Rule that listens on the external interface of the ISA Server 2004 firewall:

1. Open the Microsoft Internet Security and Acceleration Server 2004 management console and expand the server name in the left pane of the console. Click the Firewall Policy node.
2. Right-click the Firewall Policy node and point to New. Click Server Publishing Rule.
3. On the Welcome to the New Server Publishing Rule Wizard page, enter the name for the rule in the Server publishing rule name text box. In this example, we will name the rule Inbound SMTP Relay, as this rule will use the external interface of the ISA Server 2004 to accept incoming mail to be relayed. Click Next.
4. On the Select Server page, enter the IP address on the Internal interface of the ISA Server 2004 firewall that you want to publish. Enter 10.0.0.1, which is the primary IP address on the Internal interface of the ISA Server 2004 firewall machine. Click Next.
5. On the Select Protocol page, select the SMTP Server protocol from the Selected protocol list. Click Next.
   
6. On the IP Addresses page, put a check mark in the External check box and click the Address button.
7. In the External Network Listener IP Selection dialog box, select Specified IP addresses on the ISA Server computer in the selected network. Click the IP address for the external interface you want to use in the rule. In this example, the IP address is 192.168.1.70. Click Add. The IP address now appears in the Selected IP Addresses list. Click OK.
   
8. Click Next on the IP Addresses page.
9. Click Finish on the Completing the New Server Publishing Rule Wizard page.

The next step is to create the Server Publishing Rule that will accept outbound relay from the Internal network Exchange Server:

1. Open the Microsoft Internet Security and Acceleration Server 2004 management console and expand the server name in the left pane of the console. Click the Firewall Policy node.
2. Right-click the Firewall Policy node and point to New. Click Server Publishing Rule.
3. On the Welcome to the New Server Publishing Rule Wizard page, enter the name for the rule in the Server publishing rule name text box. In this example, we will name the rule Outbound SMTP Relay as this rule will use the external interface of the ISA Server 2004 to accept incoming mail to relay. Click Next.
4. On the Select Server page, enter the IP address on the Internal interface of the ISA Server 2004 firewall that you want to publish. Enter 10.0.0.10, which is the secondary IP address on the Internal interface of the ISA Server 2004 firewall machine. Click Next.
5. On the Select Protocol page, select the SMTP Server protocol from the Selected protocol list. Click Next.
6. On the IP Addresses page, put a check mark in the Internal check box and click the Address button.
7. In the External Network Listener IP Selection dialog box, select the Specified IP addresses on the ISA Server computer in the selected network option. Click the IP address on the Internal interface you want to use in the rule. In this example, the IP address is 10.0.0.10, then click Add. The IP address now appears in the Selected IP Addresses list. Click OK.
8. Click Next on the IP Addresses page.
   
9. Click Finish on the Completing the New Server Publishing Rule Wizard page.

Now we are ready to configure the SMTP Message Screener. Each Publishing Rule can be configured with a different SMTP Message Screener configuration.

Perform the following steps on the Outbound SMTP Relay Server Publishing Rule:

1. Right-click the Outbound SMTP Relay rule and click Configure SMTP.
   
2. Click the General tab in the Configure SMTP Protocol Policy dialog box. Place a check mark in the Enable support for Message Screener check box.
3. Click the Keywords tab. Place a check mark in the Enable this rule check box. Click Add. In the Mail Keyword Rule dialog box, enter resume in the Keyword text box. Select the Message header or body option. Select the Hold message option from the Action list. Click OK.
   
4. Click Apply and then click OK in the Configure SMTP Protocol Policy dialog box.

Perform the following steps on the Inbound SMTP Relay Server Publishing Rule:

1. Right-click the Inbound SMTP Relay rule and click Configure SMTP.
2. Click the General tab in the Configure SMTP Protocol Policy dialog box. Place a check mark in the Enable support for Message Screener check box.
3. Click the Keywords tab. Click the Add button. In the Mail Keyword Rule dialog box, enter mail enhancement in the Keyword text box. Select the Message header or body option. Select the Hold message option from the Action list. Click OK.
   
4. Click Apply and then click OK in the Configure SMTP Protocol Policy dialog box.
5. Click Apply to save the changes and update the firewall policy.
6. Click OK in the Apply New Configuration dialog box.

Create the Outbound SMTP Access Rule

Perform the following steps to create an outbound SMTP Access Rule that enables the ISA Server 2004 firewall to relay SMTP from the Internal Exchange Server to SMTP servers for other domains on the Internet:

1. In the Microsoft Internet Security and Acceleration Server 2004 management console, expand the computer name in the left pane of the console and click the Firewall Policy node. Right-click the Firewall Policy node, point to New and click Access Rule.
2. In the Welcome to the New Access Rule Wizard page, enter a name for the rule in the Access Rule name text box. In this example we will call this Outbound SMTP from Local Host. Click Next.
3. On the Rule Action page, select Allow and click Next.
4. On the Protocols page, select the Selected protocols option from the This rule applies to list, and click Add.
   
5. In the Add Protocols dialog box, click the Common Protocols folder and double-click the SMTP protocol. Click Close.
6. Click Next on the Protocols page.
7. On the Access Rule Sources page, click the Add button. In the Add Network Entities dialog box, click the Networks folder and double-click Local Host. Click Close.
8. Click Next on the Access Rule Sources page.
9. On the Access Rule Destinations page, click Add. In the Add Network Entities dialog box, click the Networks folder and double-click the External network. Click Close.
10. On the User Sets page, accept the default value, All Users, and click Next.
11. Click Finish on the Completing the New Access Rule Wizard page.
12. Click Apply to save the changes and update the firewall policy.
13. Click OK in the Apply New Configuration dialog box.

Configure SMTP Message Screener Logging

The SMTP Message Screener logs all messages moving the inbound and outbound SMTP relays. This logging feature helps you troubleshoot and access the e-mail messages moving through the server and confirm that the SMTP Message Screener is doing what you expect it to do.

Perform the following steps to configure the SMTP Message Screener logging feature:

1. In the Microsoft Internet Security and Acceleration Server 2004 management console, expand the computer name in the left pane of the console and click the Monitoring node.
2. Click the Logging tab in the Details pane. Expose the Task pane if it is not already open. In the Task pane, click the Tasks tab and Configure SMTP message Screener Logging.
3. In the SMTP Message Screener Logging Properties dialog box, note that the only logging format available is the File format. Select the ISA Server file format from the Format list. Confirm that a check mark appears in the Enable logging for this service check box. Click the Options button.
   
4. In the Options dialog box, confirm that ISA Logs folder is selected. Make a note of the Log file storage limits that are configured by default, and how it Maintains log storage limit by. Change the value in the Delete files older than (days) from 7 to 30. Confirm that a check mark appears in the Compress log files check box.
   
5. Click OK in the Options dialog box.
6. Click Apply and then click OK in the SMTP Message Screener Properties dialog box.
7. Click Apply to save the changes and update the firewall policy.
8. Click OK in the Apply New Configuration dialog box.

Test SMTP Filtering

Now that the SMTP Server Publishing Rule and SMTP Message Screener configurations are in place, we’re ready to test the effectiveness of the Message Screener.

Perform the following on the external client machine to test the inbound SMTP relay function:

1. On the external client computer, open Outlook Express. If presented with the e-mail account Wizard, cancel out of the Wizard so that you can manually configure the e-mail account.
2. In the Outlook Express application, click the Tools menu and click Accounts.
3. In the Internet Accounts dialog box, click Add. Click the Mail command.
4. In the Your Name text box, enter your name. Click Next.
5. In the E-mail address text box, enter an e-mail address. In this example we will enter administrator@Internal.net. Click Next.
6. On the E-mail Server Names page, confirm that POP3 is selected in the My incoming mail server is a X server list. Enter a bogus entry in the Incoming mail (POP3, IMAP or HTTP) server text box. In this example, we will enter blah.com. In the Outgoing mail (SMTP) server text box, enter the IP address that the External SMTP Relay Server Publishing Rule is listening on. In this example, the External SMTP Relay Server Publishing Rule is listening on the address 192.168.1.70, so we will enter that value into this text box. Click Next.
7. On the Internet Mail Logon page, enter a bogus account name in the Account name text box. In this example, enter the name Administrator. In the password box, enter a random password. Click Next.
8. Click Finish on the Congratulations page.
9. Click Close in the Internet Accounts dialog box.
10. Click the Create Mail button in the Outlook Express button bar.
11. In the New Message dialog box, enter the address administrator@msfirewall.org. Enter mail enhancement in the Subject text box. Click Send in the button bar.
12. Return to the ISA Server 2004 firewall machine. Click Start and Windows Explorer. Navigate to C:\Inetpub\mailroot\Badmail. You will see three files with the file extensions .BAD, .BDP and .BDR. These entries represent components of the blocked e-mail message. You can view them using the Notepad application.
13. Navigate to the C:\Program Files\Microsoft ISA Server\ISALogs folder. Double-click the ISALOG_Date_EML_xxx.iis file. Open the file with the Notepad application. There you will see entries in the log regarding how the SMTP Message Screener processed the connection.
14. You can repeat the preceding steps on the CLIENT on the Internal network. In the e-mail message, include the word resume in the subject or body of the message. You will find that message is blocked and logged by the SMTP message screener. You can also send e-mail messages without the blocked words, and the outbound SMTP relay will forward the mail to the external e-mail user.

Conclusion

In this ISA Server 2004 Configuration Guide document, we discussed how to make the ISA Server 2004 firewall your front line protection as an e-mail defense in-depth plan. The ISA Server 2004 SMTP Message Screener can provide initial inspection and protection against dangerous and inappropriate e-mail messages. The Message Screener can perform initial evaluation of SMTP messages while also providing secure SMTP relay servers that protect the mail server on the Internal network from direct connections from untrusted servers. In the next chapter of this ISA Server 2004 Configuration Guide series, we will discuss how the firewall can be used to publish an array of Exchange Server services.