Weighing IPSec Tradeoffs

  • Article

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Whether you use packet filtering or end-to-end security, deploying IPSec provides greater security at the expense of network and processing performance and compatibility with other services, tools, and features. Use the information in this section to determine the impact on your environment before deploying IPSec.

For example, you might decide that information sent between some departments, such as Accounting and Legal, require higher security than information sent to or from other departments, such as Production. You might also decide that all private information transmitted across the Internet must be secured. Finally, you might decide that you only want trusted computers to have network access to your servers. When planning your deployment, be aware that deploying IPSec on your network increases the overall cost of network and system administration. The exact cost cannot be predicted because it depends on many factors, including the level of application traffic. However, if you stage IPSec deployment in a lab, you can arrive at a good estimate. Do not deploy IPSec if the security it provides is not required. Consider whether the added cost is worth the added protection.

Slower connection establishment

IPSec is not recommended for scenarios in which clients make infrequent, short-lived connections. IPSec increases the time required to initially establish authenticated connections. Depending on policy design, network roundtrip time, and the load on the systems required to establish the connection, negotiating security initially using IKE typically increases connection time by one to three seconds. Make sure that you want to trade the one- to three-second increase in connection time for the added security. Typically, after IPSec SAs are established, no further delays occur. Because IPSec is not integrated with TCP, TCP connections can be established and ended rapidly after IPSec SAs are established.

Reduced computing performance

Although the Windows Server 2003 IPSec filtering engine is enhanced to provide significantly faster packet processing, filtering and encryption of each packet consumes computing power. As the volume of IPSec traffic — including policies, rules, and filters — increases, the load placed on the network and on computer CPUs increases also. For example, policies with 1,000 or more filters can cause a small load increase. Filters with specific source and destination IP addresses can be processed much more quickly than filters that specify a subnet or Any IP Address. IPSec implements a cache algorithm that can accelerate the processing of packets that match a filter. Packets that do not match any filter require the greatest amount of time to process.

If your systems already have high loads on their CPUs, consider adding more computers or using IPSec offload network adapters before deploying IPSec. IPSec offload network adapters accelerate cryptographic operations used to secure IPSec packets. However, these adapters do not accelerate filter processing time. IPSec-enabled network adapters can be used with Windows 2000, Windows XP, and Windows Server 2003. Such network adapters typically complete IPSec operations at such a high rate that there is minimal change in transmission speeds, resulting in minimal performance degradation. However, such adapters generally have a maximum number of SAs that they can support, after which additional SAs are processed by the Windows IPSec components.

Thwarting some networking inspection technologies

ESP and AH encapsulate IP payloads by adding a security header to every packet. Existing network management and diagnostic tools typically cannot interpret IPSec-protected packets. The tools that might be impacted include:

  • Router-based access control lists.

  • Firewall access controls.

  • Network traffic analysis and reporting tools.

  • Quality of Service (QoS).

  • Third-party load balancing devices. Windows Server 2003 IPSec supports the Windows Server 2003 Network Load Balancing service for host-based load balancing, which might provide an alternate solution.

  • Network address translation. Windows Server 2003 supports IPSec NAT-T, which is used to allow IPSec ESP-protected traffic to traverse one or more network address translators.

  • Stateful filtering controls on the direction of traffic flow. IKE and IPSec communication typically function best when traffic flows in both directions. IPSec does not easily allow for connections in only one direction.

  • Network-based intrusion detection.

These functions require that network packets be inspected, modified, or both. If IPSec protocols are not used to encrypt packets, these functions can be enhanced to inspect inside the IPSec protocol header. If IPSec encryption is required for security, however, it might be difficult to maintain full network management functionality.

Increased packet size reduces throughput, increases network utilization

IPSec protection adds overhead to IP packets, reducing effective throughput and increasing network utilization. Although the IETF design and Microsoft implementation of IPSec attempt to minimize both impacts, existing networks and servers might be operating at maximum capacity without IPSec. Before deploying IPSec, test several different IPSec policy configurations, because each policy configuration is likely to result in different performance impacts on clients and servers. It might be necessary to increase the available network bandwidth or CPU power, or install IPSec offload adapters to compensate for the increased overhead of IPSec.

Application incompatibility with IPSec NAT-T

Windows Server 2003 supports IPSec NAT-T as described by version 2 of the IPSec NAT-T Internet drafts, but some applications might not work when their traffic is first protected with IPSec and then passed through a network address translator. Test how your planned implementation of IPSec interacts with network address translators in a lab before deploying IPSec in your environment.

Loss of connectivity during cluster node addition or removal

IPSec establishes and maintains cryptographic state between computers. Many clustering and load-balancing services use the same IP address for all nodes in a cluster, which creates incompatibilities with IPSec. Windows Server 2003 IPSec has proprietary extensions that allow it to work with the Windows Server 2003 Network Load Balancing service and Windows Cluster Service. However, support for these extensions does not exist in the current Windows 2000 and Windows XP IPSec client implementations, so you might experience some loss of connectivity when you add or remove cluster nodes.

Requiring IPSec for communication between Active Directory domain members and domain controllers might block connections

IPSec is based on the authentication of computers on a network; therefore, before a computer can send IPSec-protected data, it must be authenticated. The Active Directory security domain provides this authentication using the Kerberos protocol. Accordingly, when IKE uses Kerberos to authenticate, the Kerberos protocol and other dependent protocols (DNS, UDP LDAP and ICMP) are used for communication with domain controllers. Additionally, Active Directory–based IPSec policy settings are typically applied to domain members through Group Policy. As a result, if IPSec is required from domain members to the domain controllers, authentication traffic will be blocked and IPSec communications will fail. In addition, no other authenticated connections can be made using other protocols, and no IPSec other policy settings can be applied to that domain member through Group Policy. For these reasons, using IPSec for communications between domain members and domain controllers is not supported.

ICMP-based functionality might fail

When designing an IPSec policy to secure or block ICMP traffic, keep in mind that such a policy might cause services and tools that rely on ICMP to measure network response times to produce misleading results. For example, Windows 2000 and Windows XP domain members have an IP/DNS-compatible locator, which measures ICMP response times for domain controller load balancing within a site. Also, Group Policy measures ICMP response times to determine if the connection to the domain controller is a slow link. Also, tools that depend on ICMP, such as Tracert, might not work when ICMP traffic is protected by IPSec.

Multicast and broadcast traffic might fail

In Windows Server 2003, you can configure IPSec filters to block multicast and broadcast traffic. Therefore, any application that uses multicast or broadcast might fail if the computer has an IPSec policy that blocks such traffic.