ISA Server 2000 Feature Pack 1

Archived content. No warranty is made as to technical accuracy. Content may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

Microsoft ISA Server 2000 Feature Pack 1, Version 1

With Microsoft Internet Security and Acceleration (ISA) Server, you can publish internal servers to the Internet without compromising the security of your internal network. You can configure Web publishing and server publishing rules that determine which requests will be sent to a server on your local network, providing an increased layer of security for your internal servers.

On This Page

Introduction
Publishing Scenarios
Message Filtering
Feature Pack Additions
Troubleshooting

Introduction

A common ISA Server scenario involves securing the Simple Mail Transfer Protocol (SMTP) communication of mail servers. For example, ISA Server can protect a Microsoft Exchange 2000 Server computer. The Mail Server Security Wizard configures the policy needed to allow communication between an Exchange server and the Internet. The wizard adds a set of server publishing rules, which redirect communication from Internet users at a particular port to a specified internal Internet Protocol (IP) address. The wizard also creates protocol rules that dynamically open ports for outgoing communication.

ISA Server also includes a remote procedure call (RPC) application filter, which uniquely provides an extra layer of security to the Exchange 2000 Server publishing model. This RPC application filter enables secure communication between Microsoft Outlook clients and an Exchange server, over the Internet.

Helping Secure Outlook Client Communication with Exchange 2000 Server

Outlook clients connect to Exchange 2000 Server through RPC. The RPC application filter in ISA Server protects the RPC communication, as described in this section. In this way, ISA Server protects not only POP3 and SMTP communication, but also uniquely secures RPC communication.

ISA Server's RPC application filter activates secure communication between Outlook clients and an Exchange server, over the Internet. The RPC application filter protects RPC communication over the Internet, by identifying which specific RPC interface is requested, and allowing only calls to those interfaces. Furthermore, the RPC application filter opens ports dynamically, meaning that the communication is allowed only when it is specifically requested.

In addition, Exchange 2000 Server communicates with Outlook clients using a lightweight UDP-based protocol. The RPC application filter also processes new mail notification, as follows: when an Outlook client logs on to an Exchange server, it registers to receive new mail notifications, by passing – through RPC – a port number on which it will listen. When new mail arrives, the Exchange server sends a single UDP packet to the port. To allow this type of notification, standard firewalls must typically open a wide range of ports. With the RPC application filter enabled, ISA Server intercepts registration for new mail, and dynamically opens only the necessary ports.

Thus, Exchange 2000 Server publishing is more secure with the ISA Server firewall.

For more information on notification and network address translation (NAT), see the "New Mail Notification" section.

How the RPC Application Filter Works

In an Exchange 2000 Server or Outlook client scenario, the RPC application filter works as follows:

  • The Outlook client issues a request over port 135 (TCP) through ISA Server to the Exchange server, to find the service port number associated with the Exchange RPC UUID.

  • The Exchange server sends a response back, through the ISA Server computer, to the Outlook client, with a port number on which the client can communicate. The connection to port 135/tcp is then closed.

  • ISA Server uses the RPC application filter to capture this information, and maintains it in a table.

  • ISA Server allocates a new port on the ISA Server computer, and changes the response that it sends to the Outlook client, to reflect this change. This information is also maintained in the table.

  • The Outlook client issues a request, seemingly to the Exchange server, but actually to the new port on the ISA Server computer. The ISA Server computer then sends the packet to the Exchange server. Only communication over this port is allowed.

Changing the Authentication Method

When the Outlook client connects to an Exchange server, the Exchange server instructs the Outlook client to communicate directly with a Microsoft Active Directory domain controller for authentication. In the publishing scenarios described in this document, this direct communication will not function properly if the Outlook client is on the Internet, while the domain controller is on the corporate Intranet. Because ISA Server does not publish the server running Microsoft Active Directory directory service, the Outlook client cannot contact the domain controller for authentication.

To allow this type of communication between the Internet-based Outlook client and the Intranet-based Exchange server, set the value of this registry key on the Exchange server:

HKLM\System\CurrentControlSet\Services\MSExchangeSA\Parameters

to:

Value name: No RFR Service

Value type: DWORD

Value data: 0x1

By making the registry change below, the Exchange server will proxy authentication requests to the domain controller (Active Directory server), instead of referring Outlook clients to communicate with the directory service directly. In this way, the Outlook client can authenticate itself to the domain controller, by using the Exchange server. The authentication traffic is carried out using the Exchange RPC protocol, so no additional publishing rules are required.

New Mail Notification

To enable new mail notification, Exchange 2000 Server relies on routable addressing between the Outlook client and the Exchange server. In the presence of a network address translation (NAT) device (such as ISA Server) between the client and the Exchange server, new mail notification will not function properly.

New mail notifications sent to the private address are dropped, because these notifications are unsolicited inbound UDP packets not associated with some previous outbound flow. The ISA Server Exchange RPC filter cannot access the registration payload to change the client-registered address. Currently, no NAT editor is available to access and change client-registered addresses.

The Outlook client also receives mail notifications during other communications with the Exchange server. However, if an error in any of the RPC packets occurs (this can happen when RPC is carried over the Internet), the Outlook client does not receive the new mail flag at the end of the packet. To work around this limitation, do the following:

  • For Outlook 2000, periodically press F9 to check for new mail.

  • For Outlook 2002, configure a polling interval.

Publishing Scenarios

The Exchange server that you are publishing can be installed on the ISA Server computer or on the local network. The following sections describe some Exchange 2000 Server publishing scenarios:

  • Exchange 2000 Server on local network (recommended)

  • Exchange 2000 Server on the ISA Server computer (not recommended)

Scenario 1: Exchange 2000 Server on Local Network

In this scenario, the server running Exchange is on the local network, protected by the ISA Server computer.

You can use the ISA Server Secure Mail Server Wizard to configure the ISA Server computer so that the Exchange server is available to external clients by using one or more of the following protocols:

  • Messaging Application Programming Interface (MAPI)

  • Post Office Protocol 3 (POP3)

  • Internet Messaging Access Protocol 4 (IMAP4)

  • Network News Transfer Protocol (NNTP)

  • RPC for direct access by Outlook or Exchange clients

Step 1. Create rules on ISA Server

The wizard creates rules on ISA Server, all named with the prefix Mail wizard rule.

To create rules on ISA Server

  1. The wizard creates one or more server publishing rules corresponding to each mail service that ISA Server protects. The server publishing rules created by the wizard have the following parameters:

    1. The mail server's internal IP address

    2. The external address exposed by the ISA Server computer

    3. The protocol for the selected mail service

  2. The Mail Server Security Wizard also creates protocol rules, to allow outgoing mail traffic. The protocol rules have the following parameters:

    1. The protocol is SMTP (client).

    2. Using a client address set. The client set includes the internal IP address of the Exchange 2000 Server computer.

The default gateway for the published server should point to the ISA Server computer.

Step 2. Configure name resolution for the published server

When publishing an Exchange server, you must configure a method for name resolution. There are two common methods:

  • Use an internal DNS server that forwards to a DNS server on the Internet

  • Use a DNS server on the Internet

Be sure to create a rule on the ISA Server computer that allows DNS queries for name resolution to occur.

Step 3. Configure name resolution for clients

Clients using POP3, IMAP4, SMTP, NNTP, MAPI, and Hypertext Transfer Protocol (HTTP) protocols can access the published server that is running these services, either by DNS name or by IP address. The clients must resolve the servers FQDN name by IP address, that is, the external IP address of the ISA Server computer.

You can activate this when you configure the client. When prompted for the name of the server hosting the service, type the IP address of the ISA Server computer.

If you are using DNS, you need an entry for the server hosting the services, where the IP address is the external IP address of the ISA Server computer. The DNS server will resolve the published name to the external IP address on the ISA Server computer.

Sometimes, the internal domain name differs from the public external domain name. In this case, use the public external domain name.

In particular for MAPI clients, when communicating with the Exchange server, the Outlook client specifies the fully qualified domain name (FQDN) of the Exchange server. The Exchange server responds to the client, specifying its internal name, which typically differs from the FQDN. When the client next communicates, it attempts to use the internal name, which is not recognized over the Internet. To work around this issue, set the Exchange 2000 Server computer's internal name to the same name as its FQDN.

Scenario 2: Exchange 2000 Server on the ISA Server Computer

In this scenario, ISA Server and Exchange 2000 Server are on the same computer. Note that this is not a recommended scenario, because IIS server and ISA Server have port contention on ports 80 and 443. Furthermore, when enabling the SMTP filter, there is additional port contention on port 25.

Step 1. Configure Exchange 2000 Server

To configure Exchange 2000 Server, you will disable socket pooling, provide SSL and HTTP access, and provide SMTP access.

Disabling socket pooling

To avoid port contention and to help further secure the network, configure IIS Server as described in this section. By default, socket pooling is enabled. That is, even if you configure Exchange server or IIS server not to listen on a specific port for an interface, listening will still occur on all interfaces. You must disable socket pooling for both the w3svc and smtpsvc services.

Examples

To ensure that the Exchange server listens on the specified interface, use MDUTIL.exe or ADSI to set the Metadata raw property ID numbered 1029 (DisableSocketPooling).

Example:

mdutil set -path smtpsvc/1 -value 1 -dtype 1 -prop 1029 -attrib 1

For W3svc, type the following at a command line:

adsutil set w3svc/DisableSocketPooling "True"

IIS contends with ISA Server on ports 80 and 443. When mail is enabled, it also contends with the SMTP filter on port 25. To avoid this port contention, modify the server bindings in the IIS server metabase, so that IIS server listens on these two ports on the internal IP address and on 127.0.0.1.

For example, type the following at a command line:

adsutil set w3svc/1/serverbindings <internal IP address>:80, 127.0.0.1:80

Caution: Incorrect use of this tool may damage your system. Before using this utility, back up any valued data on the computer.

SMTP Access

This section describes how to configure Exchange 2000 Server to listen for SMTP traffic only on the internal interface.

To configure Exchange Server 2000 to listen for SMTP traffic on the internal interface

  1. Open the Exchange System Manager. Click Start, click Programs, click Microsoft Exchange, and then click System Manager.

  2. In the console tree of System Manager, click Servers, click the applicable server, click Protocols, click SMTP, right-click Default SMTP Virtual Server, and then click Properties.

  3. On the General tab, click Advanced.

  4. Verify that only internal IP addresses are listed in the Address box. Remove any other addresses by selecting them and clicking the Remove button.

  5. To add the internal IP address, click Add. Then, select the internal IP address from the list. In TCP port, type:

    25

You can use the Secure Mail Server Wizard to publish the Exchange server located on the ISA Server computer. In this scenario, the Mail Server Security Wizard creates IP packet filters. IP packet filters are created for each mail service that you select. For example, imagine that you run the Mail Server Security Wizard and specify outgoing SMTP mail and POP3 client requests.

Outlook or Exchange clients cannot access the Exchange server from outside the local network using RPC connections. Only POP3 and IMAP4, also supported by Outlook, can be used.

The following IP packet filters are created:

  • An IP packet filter allowing inbound Transmission Control Protocol (TCP) connections on local Port 25 from any remote port (to allow incoming SMTP packets)

  • An IP packet filter allowing outbound TCP connections from all local ports to remote Port 25 (to allow outgoing SMTP packets)

  • An IP packet filter allowing inbound TCP connections on local Port 110 from any remote port (to allow incoming POP3 packets)

The Mail Server Security Wizard does not configure the SMTP filter when Exchange 2000 Server and ISA Server are located on the same computer. The ISA Server computer and the Exchange server must be specially configured. The following sections describe how to configure the SMTP server.

Step 2. Configure the ISA Server computer

To fully secure the co-located Exchange server, ISA Server must be specially configured by performing the following tasks:

  1. Enable the SMTP filter.

  2. Configure a server publishing rule to make the Exchange server accessible.

Enabling the SMTP filter

To enable the SMTP filter:

  1. In the console tree of ISA Server, click Internet Security and Acceleration Server, click Servers and Arrays, click the applicable array, click Extensions, and then click Application Filters.

  2. In the details pane, right-click SMTP Filter, and then click Properties.

  3. On the General tab, verify that Enable this filter is selected.

Creating a server publishing rule

Note: Do not use the Mail Server Security Wizard.

To create a server publishing rule to publish the local Exchange server:

  1. In the console tree of ISA Server, click Internet Security and Acceleration Server, click Servers and Arrays, click the applicable array, click Publishing, click Server Publishing Rules, click New, and then click Rule.

  2. Type a name for the rule, and then click Next.

  3. On Address Mapping, in IP address of internal server, type the IP address on which the Exchange server is configured to listen. In this case, this should be one of the ISA Server computer's internal IP addresses.

  4. In External IP address on ISA Server, type the ISA Server computer's external IP address. Then, click Next.

  5. On the Protocol Settings page, select SMTP Server. Then, click Next.

  6. On the Client Type page, select the clients that can access the SMTP server. Then, click Next, and then click Finish to exit the wizard.

Message Filtering

ISA Server includes message filtering functionality, which can be used to control the incoming SMTP traffic communicated through the ISA Server computer. In this architecture, ISA Server sends messages to the SMTP service, which then sends it to the message screener, which sends the messages back to the SMTP service, which may then relay it. The message filtering functionality is implemented in two components:

  • The SMTP filter is installed by default, but not enabled. It is used to specify which SMTP commands are allowed in the incoming SMTP traffic. The SMTP filter is always installed, regardless of whether you install the message screener. However, if the message screener is not installed, the filter can screen only SMTP commands and not message content.

  • The message screener is an extension to the Exchange 2000 SMTP server that can be optionally installed. During ISA Server setup, you can select to install the message screener only if an SMTP server is installed on the computer. The message screener is used to filter messages, specifying which keywords and attachments are permitted and which users and domains are denied. You configure the message screener using the SMTP filter's property sheets.

Note that although the SMTP filter and the message screener are configurable by using ISA Management, they are actually two independent entities.

ISA Server and Exchange 2000 can be deployed on the same (local) computer or on different computers. Depending on your network configuration, you will need to set up and configure SMTP server, ISA Server, the message screener, and the SMTP filter differently. This document describes the different deployment scenarios, and how you should configure ISA Server and Exchange 2000 Server for the different scenarios.

The Mail Server Security Wizard in ISA Server does not configure the SMTP filter when SMTP server and ISA Server are located on the same computer. The ISA Server computer and the SMTP server must be specially configured if you want to use the SMTP filter functionality.

For all message screener scenarios, to ensure that the message screener was installed, verify that the following registry key exists on the computer:

HKEY_CLASSES_ROOT\CLSID\{4F2AC0A5-300F-4DE9-821F-4D5706DC5B32} 

If this registry key does not exist, the message screener was not installed.

Verify that fltrnsk1.dll is in the program files\isa directory.

Scenario 1: Configuring Message Filtering for Co-located ISA Server and Exchange 2000 Server

This section describes how to configure Exchange 2000 Server and ISA Server when they are located on the same computer.

In this scenario, you should install both the message screener and the SMTP filter on the computer. When you install ISA Server and select the Full installation option, the message screener is installed with the Full install option during ISA Server setup.

The SMTP filter is always installed, but not enabled.

Scenario 2: Configuring Message Filtering for Exchange 2000 Server in Local Network

This section describes how to configure message screening when Exchange 2000 Server and ISA Server are located on different computers. In this case, two configurations are possible.

In configuration 1, the message screener is installed on the server running Internet Information Services (IIS). This means that there are actually two layers of protection:

  • ISA Server at the edge, connected to the Internet

  • IIS with the message screener, screening all incoming messages that pass through ISA Server

  • Exchange 2000 Server, on the internal network

The advantage of this configuration is that the Exchange server is further isolated from the network edge. In addition, in this scenario, the message screener screens only incoming messages, and not outbound messages.

In configuration 2, the message screener installed on the Exchange 2000 Server computer, screens all messages, both incoming and outgoing. This may cause slow performance under high loads.

You could configure Exchange 2000 Server, so that only incoming messages are screened, by adding another network adapter to the Exchange 2000 Server computer.

To configure these scenarios, perform the following steps, described in the following sections:

  1. Configure the IIS SMTP virtual server (configuration 1 only)

  2. Install the message screener, on the IIS server (for configuration 1) or on the Exchange 2000 Server computer (for configuration 2).

Step 1. Configure the IIS SMTP virtual server

For configuration 1, configure the IIS server virtual SMTP server, as described in this section.

To configure the IIS server virtual SMTP server

  1. After installing IIS with SMTP, open IIS Management and expand SMTP.

  2. Right-click Domain, and then click New Domain.

  3. In the SMTP Domain Wizard, select Remote.

  4. Type the name of the domain for which you want to accept mail; this is typically the external (public) name of the domain.

  5. After the domain is created, right-click the domain name, and then click Properties.

  6. Select Allow incoming messages to be relayed....

  7. In Route Domain, select Forward all mail to smart host.

  8. Type the IP address of the Exchange server or mail server.

Step 2. Install the message screener

This section describes how to install the message screener. In both scenarios, the message screener is installed as follows:

Note: The message screener can only be installed if Exchange 2000 Server or IIS server SMTP service is installed on the ISA Server computer.

Install the message screener only after you set up the virtual SMTP server on the IIS server.

The message screener is installed only on the first virtual server listed, as listed in Exchange System Manager or IIS.

To run the ISA Server setup in maintenance mode

  1. In Control Panel, double-click Add/Remove Programs, click Microsoft Internet Security and Acceleration Server, and then click Change.

  2. In ISA Server setup, click Continue, press the CD key, select the appropriate installation folder, and then select Custom Installation.

  3. In the Options box, verify that the ISA Services and Administration tools options are not selected.

  4. Highlight Add-in services, and then click Change Option...

  5. Select only the Message Screener option, and then click OK. Then, finish the setup process, selecting the default options.

Note: These options are also available when you install ISA Server and specify Custom installation.

Scenario 3: Configuring ISA Server

On the ISA Server computer, perform the following steps:

  1. Enable the SMTP filter.

  2. Create appropriate server publishing rules, to publish the Exchange server (or IIS server, if you are publishing an IIS server).

  3. Configure the SMTP filter, to screen appropriately.

Using the SMTPCred Tool

If you are running ISA Server Standard Edition in either of these configurations, or if you are running ISA Server Enterprise Edition as a stand-alone server, run the SMTPCred.exe tool.

When you run the tool, enter credentials that have administrator privileges on the ISA Server computer. The user account required for SMTPCred.exe does not require special rights, but it should be in the domain.

SMTPCred.exe is available on the ISA Server CD in the .\isa\i386 folder.

Scenario 4: Distributed COM and ISA Server

The ISA Server SMTP filter transmits data over Distributed COM (DCOM). Verify that DCOM is working properly between ISA Server and the server where the SMTP message screener is installed. Also, consider carefully the security implications of using DCOM when configuring it.

To configure DCOM on ISA Server

  1. Open the Distributed COM Configuration utility by typing dcomcnfg.exe at a command prompt.

  2. On the Applications tab, select VendorData class, and then click Properties.

  3. On the Security tab, select Use custom access permissions, Use custom launch permissions, and Use custom configuration permissions.

  4. For each of these permission settings:

    1. Click Edit.

    2. In Registry Value Permissions, click Add, select Everyone, and then click OK.

    3. On Registry Value Permissions, in Type of Access, select
      Allow Access when setting access permissions,
      Allow Launch when setting launch permissions,
      Full Control when setting configuration permissions.

This grants access permission to internal users only, who are familiar with the Vendor Class's program ID, to add COM objects to the ISA Server computer.

Scenario 5. Relay Considerations

The Exchange server can be used as a relay for inbound and outbound SMTP traffic. For maximum security, you should use Exchange 2000 Server as the endpoint server in your organization with the message screener installed on this computer. In this case, configure the relay option to allow only computers and domains in your organization.

To configure the relay option for Exchange 2000 Server

  1. To open the Internet Services Manager, click Start, click Programs, click Microsoft Exchange, and then click System Manager.

  2. In the console tree of System Manager, click System Manager, click Servers, click the applicable server, and then right-click Default SMTP Virtual Server.

  3. Click Properties.

  4. On the Access tab, click Relay.

  5. In Relay Restrictions, select Only the list below. Add the computers and domains in your organization.

If you are using other mail servers for client mailboxes, the server with the message screener (virtual SMTP server) should be configured to route the SMTP traffic to those servers. For more information on routing configuration, refer to Microsoft SMTP service in Microsoft Windows Help.

Scenario 6: Administering the SMTP Filter

You can use ISA Management to configure the SMTP filter and the message screener.

Configuring the SMTP filter

To configure the SMTP filter and the message screener

  1. In ISA Management, click Internet Security and Acceleration Server, click Servers and Arrays, click the applicable array, click Extensions, and then click Application Filters.

  2. In the details pane, right-click SMTP Filter, and then click Properties.

  3. To configure the message screener, click the Attachments tab or the Keywords tab and set the fields appropriately. For more information, see ISA Server Help.

  4. To configure the SMTP Filter, click the Users/Domains tab or the SMTP Commands tab and set the fields appropriately. For more information, see ISA Server Help.

Note: For ISA Server Enterprise Edition, when ISA Server is installed as an array member, you must have permissions to change the enterprise configuration, to modify and configure the SMTP filter. This is because the SMTP filter applies to all the arrays in the enterprise.

Feature Pack Additions

ISA Server Feature Pack 1 includes an enhancement to Exchange 2000 Server RPC publishing, whereby administrators can enforce encryption of RPC communication between the Exchange server and Outlook clients. When this feature is activated, RPC traffic between Outlook clients and the Exchange server is blocked, if the traffic is not encrypted.

The Publish Outlook Web Access Server Wizard introduced in ISA Server Feature Pack 1 facilitates Outlook Web Access publishing by automatically generating the necessary Web publishing rule and associated destination sets.

For more information about ISA Server Feature Pack 1, see the ISA Server Feature Pack 1 Web site (https://go.microsoft.com/fwlink/?LinkID=11922).

Troubleshooting

This section contains troubleshooting information, tips, and hints to help you configure Exchange 2000 Server publishing and SMTP filter scenarios.

DCOM/RPC Communication

Some communication problems may result from improper configuration of DCOM. This section describes how to troubleshoot DCOM configuration issues.

Configuring DCOM

Before configuring DCOM, use Task Manager to verify that ISA Server services are not running. All settings are assigned to a process when it starts. If the server is running when you modify the settings, the new settings will take effect the next time that the server is launched.

  1. Verify Dcomcnfg is configured correctly:

    1. Open DCOMCONFIG (Click Component Services, click Computers, click My Computer, and then click DCOM Config).

    2. For the VendorData class, right-click, and then click Properties.

    3. On the Security tab, for Launch, Access, and Configuration Permissions, verify that Customize is selected and that, under permissions, Everyone is added.

  2. After you reconfigure Dcomcnfg, reboot the ISA Server computer and the computer running the message screener.

  3. On the computer on which message screener is installed, use smtpcred to check if the client is using the fully qualified domain name (FQDN) or the server name. If either is being used, replace it with the IP address of the server.

    If this doesn't resolve the issue, check the network configuration.

Auditing to determine DCOM communication problems

You can use the Event Viewer to find additional auditing information for why the DCOM connection failed. However, logging these types of events is usually not enabled by default. You need to set the auditing options to activate logging. In Windows 2000, activate these options as follows:

  1. On the Start menu, click Programs, click Administrative Tools, and then click Local Security Policy.

  2. In Local Security Settings, click Security Settings, click Local Policies, and then click Audit Policy.

  3. In the details pane, right-click and then click Properties for each of the following:

    1. Audit logon events

    2. Audit object access

    3. Audit privilege use

  4. For each of the selected policies, in the Properties dialog box, select Success and Failure.

After configuring auditing, try to use DCOM communication. If you still get error messages, use Event Viewer to see if there are any DCOM events. The event may explain why access was denied. Furthermore, it includes information about:

  • Who is logged on to the client computer

  • Whether the user is a domain user or local user

  • Whether the protocol requested by the client is available on the server

COM logs are usually added to the system log.

Publishing Rules

Ensure that there are server publishing rules and that you have the appropriate IP address for your mail server.

  • If you have message screener on your IIS server, the IP address should point to your IIS server.

  • If you have message screener on your Exchange server, the IP address should point to your Exchange server.

Message Screener Computer

The message screener might be installed on an IIS server or on an Exchange server. Verify the following:

  • Verify that Smtpcred has the appropriate credentials. SMTPCred is an account with administrator privileges on the ISA Server computer.

  • Verify that the message screener loads dynamically. You can check if it loaded correctly by doing the following:

    • Send a mail message with data. Then, at a command prompt, type tlist -m fltrsnk1.dll. It should show the task where this is loaded, which in our case is inetinfo.exe.

      Note: If a filter is not in use for a period of time, it will unload.

  • Verify that the message screener registry key was configured properly:

    SMTP hotfix:

    Windows Registry Editor Version 5.00

    [HKEY_CLASSES_ROOT\CLSID\{4F2AC0A5-300F-4DE9-821F-4D5706DC5B32}\InprocServer32]

    "ThreadingModel"="Both"

    ISA RTM:

    Windows Registry Editor Version 5.00

    [HKEY_CLASSES_ROOT\CLSID\{4F2AC0A5-300F-4DE9-821F-4D5706DC5B32}\InprocServer32]

    "ThreadingModel"="Free"

Exchange Specific

Follow the guidelines in this section to troubleshoot Exchange 2000 Server configuration:

  1. Ensure that name resolution is working correctly and that the Exchange server can resolve names correctly.

    1. Verify the DNS configuration. Exchange should point to an internal DNS server that then forwards to an external DNS server.

    2. In Exchange MMC Admin under the server->protocols node for the properties of POP3 and SMTP, you will see an authentication key. Verify the authentication key setting. For SMTP, Anonymous authentication should be set. This allows external SMTP servers to communicate with your mail server.

  2. For this scenario, an SMTP connector should be configured. Ensure that the configuration is correct.

Network or SMTP Basic Functionality

This section describes how to verify that SMTP and the network are functioning properly. Do the following:

  1. Use Telnet to port 25 with an external client to issue a command, for example: hello.

  2. If step 1 works, place a sniff on external and internal. Verify that packets are being passed back and forth.

  3. Use a Netmon trace between the ISA Server computer and the computer running the message screener. Verify that there are RPC calls being made to the ISA Server computer. Look for protocol MSRPC. You should not see ISA Server responding with 80070005, which is an access_denied response. An example from a sniff:

00000030 05 00 00 83 10 00 00 00 E0 00 .......... 00000040 10 00 E5 03 00 00 9C 00 00 00 00 00 07 00 11 00 ................ 00000050 00 00 3C 04 00 00 D0 06 00 00 65 03 00 00 05 00 ..<.......e..... 00000060 06 00 00 00 00 00 00 00 00 00 6E C2 31 60 B9 24 ..........n.1`.$ 00000070 58 44 8D 74 80 0C C5 99 E2 2D 00 00 00 00 55 73 XD.t.....-....Us 00000080 65 72 26 00 00 00 4C 00 00 00 26 00 00 00 7B 00 er&...L...&...{. 00000090 30 00 39 00 63 00 34 00 66 00 37 00 36 00 65 00 0.9.c.4.f.7.6.e. 000000A0 2D 00 35 00 38 00 64 00 38 00 2D 00 34 00 37 00 – .5.8.d.8.-.4.7. 000000B0 34 00 34 00 2D 00 38 00 62 00 64 00 66 00 2D 00 4.4.- .8.b.d.f.-. 000000C0 33 00 65 00 62 00 33 00 30 00 32 00 66 00 36 00 3.e.b.3.0.2.f.6. 000000D0 61 00 37 00 62 00 30 00 7D 00 00 00 00 00 00 00 a.7.b.0.}....... 000000E0 00 00 00 00 00 00 67 00 72 00 6F 00 75 00 70 00 ......g.r.o.u.p. 000000F0 2E 00 63 00 6F 00 6D 00 03 00 28 00 6A 00 0A 02 ..c.o.m...(.j... 00000100 04 00 70 C9 11 00 01 00 00 00 00 00 00 00 00 00 ..p............. 00000110 00 00 00 00 00 00 ......
Response- MSRPC: c/o RPC Fault: call 0x3E6 context 0x0 status 0x80070005 cancels 0x0

New Service Pack 1 Features

ISA Server 2000 Service Pack 1 (SP1) introduced functionality to set a registry value that causes ISA Server to replace the source address of incoming requests, so that the packets that are sent to the internal server have the source address of the ISA Server computer. This allows the normal IP routing configuration in large networks to route these packets back to the ISA Server computer. ISA Server, in turn, can perform address translation on these packets, when responding to the original external host, where the request originated.

Before this new functionality was introduced, when incoming packets were received by ISA Server, the destination address was changed when the request was forwarded to the published server. The original destination address was the external IP address of the ISA Server computer, and the new destination address was the IP address of the internal published server. However, the packet sent from the ISA Server computer to the published server still had the original source address of the external client, where the packet originated. For the published server to return a response to the client, it required a default route to the Internet, through ISA Server. Some network topologies (particularly in large corporate networks) do not have default routes to the Internet. In those environments, the published server could not respond to the client.

The new SP1client address translation feature resolves this issue, by replacing the client address with the address of the ISA Server computer.

To enable client source address translation

  1. In the Registry Editor, locate this registry key:

HKEY_LOCAL_MACHINE\system\currentcotrolset\services\fwsrv\parameters

  1. On the Edit menu, click Add Value, and then add the following registry value:

    Value name: UseISAAddressInPublishing

    Data type: REG_DWORD
    Radix: Binary
    Value data: 1

  2. Restart the Firewall service.

Note: This feature works only if the published protocol does not require an application filter that is there were no secondary connections in the protocol. This feature also works only for publishing FTP and RPC servers because only FTP and RPC application filters have this support.