Securing Communication: Front-End to Other Servers


Topic Last Modified: 2005-05-24

HTTP, POP, and IMAP communication between the front-end server and any server with which the front-end server communicates (such as back-end servers, domain controllers, and global catalog servers) is not encrypted. When the front-end and back-end servers are in a trusted physical or switched network, this is not a concern. However, if front-end and back-end servers are kept in separate subnets, network traffic may pass over unsecured areas of the network. The security risk increases when there is greater physical distance between the front-end and back-end servers. In this case, it is recommended that this traffic be encrypted to protect passwords and data.

Windows supports IPSec, which is an Internet standard that allows a server to encrypt any IP traffic, except traffic that uses broadcast or multicast IP addresses. Generally, you use IPSec to encrypt HTTP traffic; however, you can also use IPSec to encrypt all traffic.

With IPSec you can:

  • Configure two servers that are running Windows to require trusted network access.

  • Exchange data that is protected from modification (using a cryptographic checksum on every packet).

  • Encrypt any traffic between the two servers at the IP layer.

In a front-end and back-end topology, you can use IPSec to encrypt traffic between the front-end and back-end servers that would otherwise not be encrypted.

The method in which data is secured using IPSec depends on which protocol is used: Authentication Header (AH) or Encapsulating Security Payload (ESP). With AH, the packets are not encrypted; AH adds a checksum to the IP packet. AH guarantees that the packet came from the expected host, was not impersonated, and was not modified in transit. AH uses IP protocol 51. ESP, which uses IP protocol 50, encrypts the entire contents of the IP packet. Both forms of IPSec provide a reliable and trusted communication channel that an attacker cannot easily insert data into or interrupt.

IPSec encryption affects the performance on both the front-end and back-end servers; the precise extent to which it affects performance, however, depends on the type of encryption used.

You should configure IPSec on the back-end servers so that they respond appropriately when they receive a request for IPSec communication. However, the back-end servers should not require that all communication from all clients be encrypted using IPSec.

Windows has three IPSec policies installed by default. Select the "Client (respond only)" policy for the back-end server. With this policy enabled on the back-end server, the front-end server can use IPSec to communicate safely with the back-end server, while other clients (including earlier versions of MAPI clients like Microsoft® Office Outlook® 2002) and servers can communicate with the back-end server without needing to use IPSec.

When a firewall or filtering router is used between the front-end and back-end servers, the filters must allow IPSec to pass through it.

IPSec does not work if there is a Network Address Translation (NAT) server between the perimeter network and the corporate network.

When using IPSec, configure the ports as follows:

  • HTTP (TCP port 80)   HTTP (TCP port 80) is no longer required and should be blocked.

  • Other protocol ports   If you are using IPSec to encrypt all traffic, you do not need other protocol ports (except possibly Kerberos).

  • 500/UDP   Open the IPSec negotiation port at 500/UDP, which is required. 500/UDP is the well-known Internet assigned port for the Internet Key Exchange (IKE) Standard, which is defined in RFC 2409. This standard uses UDP and not TCP/IP to avoid the security weaknesses of relying on TCP/IP connections. IKE provides strong security for the negotiation data packets. It establishes and maintains the IPSec connections, named security associations. 

  • IP protocol 50 or 51   Allow either IP protocol 50 (AH) or IP protocol 51 (ESP), depending on the protocol you are using.

  • UDP port 88 and TCP port 88   Kerberos traffic uses UDP/TCP protocol source and destination port 88. Kerberos is itself a security protocol, and was therefore not included by IPSec filters in Microsoft® Windows 2000 Server, even if the filter included the IPSec ports. Therefore, if you use IPSec on Windows 2000 Server you still need to open UDP port 88 and TCP port 88. Alternatively, you can force Kerberos to be included in an IPSec filter by setting the "NoDefaultExempt" registry key. In Windows Server 2003, Kerberos will be included by IPSec filters by default. If Kerberos is included in the IPSec encryption you cannot use Kerberos to authenticate the IPSec session itself. You need to use either a certificate or a preshared key

  • For more information about configuring IPSec with firewalls, see Microsoft Knowledge Base article 233256, "How to Enable IPSec Traffic Through a Firewall."

  • For more information about IPSec and Kerberos, see Microsoft Knowledge Base article 810207, "IPSec Default Exemptions Are Removed in Windows Server 2003."

  • In a perimeter network, you may want the increased security of IPSec without losing the ability to filter traffic for security reasons. In this case, you can set up an IPSec tunnel from the front-end server to the internal firewall, and then a separate tunnel from the internal firewall to the back-end server. This approach secures the information about the wire while allowing you to use filtering or intrusion detection software or techniques on the traffic before it reaches your internal network.

IPSec encryption occurs after the application (in this case, Exchange) passes the request to Windows to send to the server. So, as far as Exchange is concerned, the request is made over HTTP using TCP/IP port 80. However, before the traffic leaves the server, it is intercepted, possibly encrypted if ESP is configured, and sent over a separate channel (IP protocols 50 or 51). Therefore, the encryption is transparent to the Exchange applications that are running on each server, and the fact that the data never used port 80 is not an issue to these applications.