Network Load Balancing Integration Concepts for Microsoft Internet Security and Acceleration (ISA) Server 2006

Microsoft Internet Security and Acceleration (ISA) Server 2006 provides Network Load Balancing (NLB) integration used to configure and manage the NLB functionality of Microsoft Windows Server® 2003 running on ISA Server arrays. When you configure NLB through ISA Server, NLB is integrated with ISA Server functionality. This provides important functionality that is not available in Windows NLB alone.

In addition, ISA Server monitors NLB configuration, and discontinues NLB on a particular computer as necessitated by its status. This prevents the continued functioning of NLB when the state of the computer does not allow the passage of traffic. For example, if there is a failure of the network adapter on the computer, or if you stop the Microsoft Firewall service, ISA Server stops NLB-directed traffic from passing through that computer. When the issue is resolved, ISA Server will again allow traffic to pass through that computer.

ISA Server works with Windows NLB to automatically configure bidirectional affinity, and does so for multiple networks. This guarantees that traffic is handled in both directions by the same array server.

This document describes ISA Server 2006 NLB integration.

NLB Integration

Network Load Balancing (NLB) enables all cluster hosts on a single subnet to concurrently detect incoming network traffic for the cluster Internet Protocol (IP) addresses. On each cluster host, the NLB driver acts as a filter between the network adapter driver and the TCP/IP stack to distribute the traffic across the hosts. ISA Server takes over at this point, enabling NLB in complex deployment scenarios, including virtual private networking, Cache Array Routing Protocol (CARP), and Firewall Client.

By enabling integrated NLB on an array of ISA Server firewalls, you establish the framework for NLB configuration at the network level. That is, ISA Server load balances traffic on a per-network basis. After you enable NLB on the specific networks that you want load balanced, ISA Server determines the network adapter that will be used for that network. If there is more than one network adapter available, ISA Server selects the network adapter based on name in alphabetical order.

ISA Server performs stateful inspection on all traffic. For this reason, ISA Server works with Windows NLB to ensure that incoming and outgoing traffic for each session is handled by the same array member. This is important, because this enables ISA Server to perform stateful inspection on the traffic.

Benefits of ISA Server Integrated NLB

• Integrated NLB provides high availability and scalability of servers using a cluster with multiple host computers working together.
• Integrated NLB delivers high availability by redirecting incoming network traffic to working cluster hosts if a host fails or is offline. Existing connections to a failing host are lost, but the services remain available. In most cases, client software automatically retries the failed connections, and the clients experience a delay of only a few seconds in receiving a response.
• Integrated NLB delivers scaled performance by distributing the incoming network traffic among hosts, using one or more virtual IP addresses (the cluster IP addresses) assigned to the NLB cluster. The hosts in the cluster then concurrently respond to different client requests.
• Integrated NLB employs a fully distributed algorithm to statistically map incoming clients to the cluster hosts based on their IP addresses. When inspecting an arriving packet, all hosts simultaneously perform this mapping to quickly determine which host should handle the packet. Although the mapping changes when the number of hosts changes, NLB continues to maintain the existing TCP connection.
• Integrated NLB incorporates the benefits of NLB into the product. When integrated NLB is enabled, each network in an array can be configured as an NLB cluster. The NLB algorithm is applied to the network, so that the load is balanced across the NLB-enabled network.

NLB Integration Modes

Network load balancing can be configured using ISA Server in NLB integrated mode, or through Windows Server 2003. NLB for an array can be configured in one of these modes:

• Integrated NLB. In this mode, you use ISA Server Management to configure NLB, and to specify whether a specific network should be load balanced. In this mode, you have the benefits of integrated easy-to-manage configuration, array integrity maintenance, multi-networking, virtual private network (VPN) support, and troubleshooting information. When you enable NLB integration, the NLB configuration for the following is supported: unicast mode and single affinity.
• Non-integrated NLB. In this mode, you use the Windows-based NLB tools to configure NLB. Non-integrated NLB does not provide the benefits of ISA Server integrated network load balancing.

Enabling Integrated NLB

By default, NLB integration is not enabled when you install Microsoft ISA Server 2006. After NLB integration is enabled for the array, you can enable and configure NLB on the array-level networks.

You can configure ISA Server to work in integrated NLB mode for any array that contains more than one array member. After you enable NLB integration on an array, we recommend that you configure and enable NLB for each connected network in the array, except the intra-array network. The intra-array network is used for communication between array members and should not be load balanced. In additional, you should enable NLB on the network that is used to connect to the disconnected network, and is physically connected to ISA Server. For example, if the disconnected network is behind the External network, then NLB should be enabled on the External network.

When you enable NLB integration for an array, you can monitor NLB for each array member, draining or stopping NLB on a specific array member as appropriate.

Using NLB Without ISA Server NLB Integration

If integrated NLB mode is not enabled for an array, NLB can be configured for the operating system using Windows-based NLB tools. However, if you choose to configure NLB without ISA Server NLB integration, none of the specific benefits of ISA Server load balancing will be available.

If you configure NLB using ISA Server integration, and then disable integrated NLB mode, the existing Windows NLB settings remain configured on each array member. You can use Windows NLB tools on each array member computer to further modify these NLB configuration settings.

Selecting the Network Adapter

When you configure integrated NLB mode, ISA Server load balances traffic on the specified networks. After you enable NLB on a specific network, ISA Server automatically determines which adapter in the network should be used for NLB. There can be only one such adapter per network.

ISA Server uses the following algorithm to select the network adapter to use for NLB:

1. Identifies all the network adapters in the load balanced network that have a static IP address configuration. These are network adapters that are not assigned by Dynamic Host Configuration Protocol (DHCP).
2. Alphabetizes the network adapters by name.
3. Selects the first network adapter that has an IP address on the same subnet as the specified virtual IP address.

This means that you can control which network adapter will be selected by renaming the adapter, and then restarting the Microsoft Firewall service.

Network Rules and NLB

Network rules define the relationship between networks. The relationship can either be defined as network address translation (NAT) or route.

When traffic is configured as NAT from one network to another, the dedicated IP address is used for outgoing communication. The traffic always returns to the same array member.

When traffic is configured as routed, the destination computer is provided the source computer IP address. For this reason, the destination computer must use the virtual IP address as its default gateway when communicating. Because NLB is integrated with ISA Server, NLB is designed to choose the appropriate array member.

Virtual IP Addresses and Dedicated IP Addresses

When you configure NLB for a network, you must specify at least one virtual IP address for the network. With NLB integration enabled, ISA Server modifies both the network properties and the TCP/IP properties of the network adapter. Using ISA Server Management, you can configure more than one virtual IP address for each load balanced network. In some scenarios, such as NLB publishing scenarios, you might want to have multiple virtual IP addresses.

Each network adapter on each array member has a dedicated IP address. The dedicated IP address is actually the original primary IP address on the network adapter of the NLB-enabled network. In integrated NLB mode, each network adapter must have its own dedicated IP address.

The network adapter used for NLB must have a dedicated IP address. Without this dedicated IP address, communication between array members and other hosts through this network adapter might not work properly. The dedicated IP address and the virtual IP address must belong to the same subnet and have the same subnet mask.

When you specify the virtual IP address, ISA Server sets this IP address as the second IP address of all the network adapters attached to the applicable network in the array. ISA Server also sets the first IP address in the TCP/IP properties of these network adapters as the dedicated IP address, if it is not the same as the virtual IP address.

When you add IP addresses to your network adapters through the ISA Server NLB properties, each adapter on the ISA Server array will be assigned an identical IP address in addition to the dedicated IP address of the adapter on that server. For example, if you add the IP address 206.73.118.1 to the External network, each array member's external network adapter will have that address added. This eliminates the risk of non-matching IP addresses, which can occur when manually assigning addresses to each array member's adapter.

The primary virtual address should not be listed in the list of additional virtual IP addresses.

Deploying NLB With ISA Server

Depending on the specific deployment scenario, ISA Server performs NLB differently. This section describes how to deploy and configure NLB for various topologies.

Configuring NLB When Joining a Server to an Array

When you set up ISA Server, joining the server to an existing array, ISA Server enables NLB integration for this new array member if NLB is enabled for the entire array. However, you may not want the NLB service to start automatically when installation completes for the following reasons:

• Existing connections may be stopped when a new array member joins the NLB cluster. For this reason, you might want to schedule starting NLB on the new array member only when there is relatively little network activity.
• You might want to review the new array member configuration before joining it to the NLB cluster. Thereafter, you can start NLB on that array member.

To ensure that ISA Server does not automatically start NLB on the new array member, create a batch file with a single command nlb suspend executed in a loop. Execute this batch file repeatedly as you install the new array member.

CARP and NLB Integration

NLB and CARP both improve performance across array servers, each offering different load sharing functionality.

NLB integration allows for load balancing of all types of network traffic going into the array. Unlike CARP, it is not limited to Web requests. NLB uses virtual IP addresses that are shared among all array servers. Each array member can pick packets sent to a virtual IP address, according to the NLB algorithm. To send anything to a specific array member, a dedicated, non-virtual IP address must be used as the destination.

CARP does not include the virtual IP address in the script sent to clients or when server-side requests are forwarded to other array members. Instead, the specified dedicated IP address on the network is used. CARP shares the cache disk among array members, which allows a larger number of objects to be stored in the cache.

CARP and NLB can be both be enabled on the same network, however, this should only be done in organizations where browsers use automatic detection or a configuration script.

Publishing and NLB

When you configure NLB for arrays with published servers, the network is load balanced according to the IP addresses of the clients, rather than according to the IP address of the published server.

For example, suppose that you publish a server, which is located on an NLB-enabled network, to clients that are located on a different NLB-enabled network. When clients access the published server, the load is balanced according to the clients' IP addresses, and not according to the published server's IP address.

Multiple Virtual IP Addresses

When NLB integration is enabled, ISA Server enables you to add additional virtual IP addresses to network adapters across your array. These additional IP addresses can be used in NLB publishing scenarios.

For example, consider a mail publishing scenario, where you may want to configure one virtual IP address on the External network for one mail server and a second virtual IP address for an additional mail server.

In another scenario, you may require more than one virtual IP address when you are publishing two Web servers to two distinct public names on the Internet, such as https://www.contoso.com and https://www.fabrikam.com. Two Web listeners are required, each with its own digital certificate matching the public site name. Because both listeners will listen on port 443, they must listen on separate virtual IP addresses. The addition of virtual IP addresses makes this possible.

Multiple Network Adapters and NLB

In a complex network topology with NLB enabled, multiple ISA Server array members may be connected through more than one network adapter, and the Configuration Storage server may be installed on one of the array members. In this case, a request to connect from an array member to the Configuration Storage server may fail.

In this scenario, to ensure that the connection requests do not fail, perform the following steps:

1. Create a new Domain Name System (DNS) entry pointing to the IP address on the intra-array network of the Configuration Storage server.
2. Register the intra-array name in the Kerberos database using the Setspn.exe tool.
3. Change the array properties to use the new DNS entry.

For example, in the scenario where the Configuration Storage server is installed on a computer named fw1.contoso.com, register fw1a.contoso.com, where fw1a.contoso.com is pointing to the intra-array IP address of the Configuration Storage server.

To register the new name in the Kerberos database, run these commands:

• setspn -a ldap/fw1a.contoso.com FW1
• setspn -a ldap/fw1a.contoso.com:2171 FW1

Modify the Configuration Storage server array property, to fw1a.contoso.com.

Configuring VPN with NLB Integration

Remote Access VPN and NLB

When a remote access client initiates a VPN connection to an array, one of the array members establishes the VPN connection and allocates an IP address for that client. From then on, all traffic for that remote client passes through that array member.

Site-to-Site VPN and NLB

When NLB is enabled on a remote site network, ISA Server automatically assigns one array member to handle the VPN tunnel. In this way, parallel tunnels between two sites are not created. The tunnel is re-created on another server if the assigned server fails for any reason.

Two arrays might communicate using site-to-site VPN tunnels. However, each site must know the dedicated IP address of the other site. For Point-to-Point Tunneling Protocol (PPTP) and Layer Two Tunneling Protocol (L2TP) remote site networks, you must specify the dedicated IP addresses of the remote sites as additional remote gateways.

You can use the NLB functionality of ISA Server to configure and manage the NLB functionality of Windows Server 2003 running on ISA Server arrays. When the remote site network is an NLB-enabled array, the initial connection from this array of ISA Server firewalls will be to the virtual IP address of the computer. The tunnel will be established from one of the dedicated IP addresses on the remote array. For this reason, you must specify all the dedicated IP addresses as additional remote tunnel endpoints.

In addition, the following is supported in integrated NLB mode, in a site-to-site VPN deployment:

• Automatic routing of client requests to the array member that is hosting the VPN connection.
• Automatic shifting of the connection to another ISA Server array member, if the server that owns a site-to-site VPN connection fails.

NLB maintains existing PPTP and Internet Protocol security (IPsec) tunnel connections. This implies that in VPN scenarios, even if the mapping changes when the number of hosts changes, NLB will continue to maintain the tunnel.

Be sure to consider the following requirements when configuring a VPN site-to-site network:

• If you are using a multi-server ISA Server array, and plan on using NLB, you must use ISA Server integrated NLB. If you use Windows NLB, site-to site connectivity will not be supported.
• If you are not using ISA Server to provide NLB functionality, you must configure your corporate routers to make sure that traffic from clients assigned to a particular pool of a particular computer running ISA Server services is routed back through that server.
• If you do configure NLB on the ISA Server array that provides the static addresses, the routing of client traffic is handled automatically by ISA Server. In this case, configure your routers to use the ISA Server array’s virtual IP address for all static routes.
• When you use ISA Server integrated NLB, it selects a server for each site-to-site connection, and provides failover protection for that connection. When NLB is enabled, NLB must be configured on the External network for site-to-site connections to function properly. In addition, NLB should be enabled on each network with which the remote site network has a route relationship.
• In a multi-server ISA Server array, where NLB is enabled, we recommend that you do not install the Configuration Storage server on one of the array members. When a Configuration Storage server is installed on an array member, and that array member does not handle the site-to-site connection, the remote site will lose connectivity with the Configuration Storage server. Install the Configuration Storage server on a separate computer behind the ISA Server array.

Connection Owner

When a site-to-site connection is established with an array of ISA Server computers, only one array member is actually the connection owner. The connection owner is the VPN tunnel endpoint.

When NLB is enabled, ISA Server automatically assigns the connection owner. No additional configuration is required. ISA Server uses an algorithm to optimize the connection owner assignment, creating as balanced a network as possible. After a tunnel has been established, the server assigned as the connection owner does not change, even if other servers are added or removed. If the assigned connection owner becomes unavailable, ISA Server automatically passes the connection to another array member. In this way, ISA Server supports failover for VPN site-to-site connections.

When NLB is not enabled, you must assign a connection owner for the remote site network. If the connection owner becomes unavailable, there will be no connectivity to the remote site.

IP Addresses for NLB-Enabled Remote Sites

When the remote site network to which you are connecting is an array with NLB enabled, the address you specify for the remote tunnel endpoint must be the virtual IP address of the NLB-enabled array.

When you create the remote site network, specify all the addresses in the remote site.

In addition, for IPsec networks, for HTTP proxy or NAT traffic between sites, you must include all the dedicated IP addresses of the network adapters associated with the remote site network. This is because the source IP addresses for HTTP proxy and NAT traffic from remote sites are subject to address translation (on the remote side). The local site therefore sees the traffic as if it is arriving from the primary IP address of the remote site—from its dedicated IP address.

Monitoring NLB Status

In integrated NLB mode, ISA Server checks that all the servers in the NLB cluster are fully operational and that the traffic can flow appropriately through the various networks. Further, in integrated NLB mode, ISA Server is responsible for ensuring that either all NLB clusters are operational or that no NLB clusters on the specific array member are operational.

You can monitor NLB for each array member, draining or stopping NLB on a specific array member as appropriate.

If NLB is not functioning on one network on a server, ISA Server will stop NLB on all the networks on this server. When the server becomes available, ISA Server adds it back to the array. Specifically, ISA Server determines that an array member is available for NLB clustering when the following are true:

• The ISA Server computer is available.
• The Microsoft Firewall service is running.
• NLB is successfully configured on all network adapters of all NLB-enabled networks.

On each server, NLB is configured to start only when the Firewall service on that server is running.

In integrated NLB mode, NLB status can be one of the following:

• Not configured. This status indicates that there are no NLB-enabled networks.
• Configuring. This status indicates that ISA Server is currently configuring the array member, or changing its NLB state.
• Running. This status indicates that NLB is running on the specific array member server. The server will be subject to the NLB algorithm for load balancing.
• Draining/Stopping. This status indicates that only active connections will be served by this array member. When load balancing all future connections to the array, this server will not be included in the NLB algorithm. Note that when you drain/stop an array member, the status will always indicate Draining/Stopping.
• Stopped. This status indicates that no connections are served by this array member.
• Stopped due to a local problem. This status indicates that ISA Server cannot configure NLB, either due to a configuration problem, or due to an error that occurred when previously configuring NLB.
• Stopped due to a VPN problem. This status indicates that NLB was stopped because the Routing and Remote Access service is not responding or because no address pool is configured for the array member.
• Suspended. This status indicates that NLB is stopped and will remain stopped even if ISA Server or the ISA Server computer is restarted.
• Unavailable. This status indicates that ISA Server cannot determine the status of the NLB service on the array member.

Using ISA Server Management, you can manually suspend, stop, drain, or start the NLB service for each server in the array. For example, to remove a server from the NLB cluster, stop the NLB service on that server. In the situation where you want to suspend NLB while you do some maintenance work on a member server, suspend the NLB service on that server. Then, when you complete maintenance, resume NLB, and the member server is again considered when balancing the load.

Best Practices

Follow these guidelines when configuring and deploying NLB-enabled networks with ISA Server:

• ISA Server NLB is based on the NLB features of Windows Server 2003. Follow the Windows Server 2003 capacity guidelines when deploying NLB for ISA Server. For more information, see Network Load Balancing Clusters: Capacity Planning at the Microsoft TechNet Web site.
• For security reasons, we recommend that you protect the NLB cluster with a router. For more information, see Network Load Balancing: Security Best Practices for Windows 2000 and Windows Server 2003 at the Microsoft TechNet Web site.
• Carefully plan how you use network devices to connect array members. For more information, see the Microsoft Knowledge Base articles at Microsoft Help and Support.
• If the Configuration Storage server is installed on an array member (a computer with ISA Server services), configure the name of the Configuration Storage server to resolve to the intra-array address of the array member.
• When NLB is enabled, it synchronizes array members by using pure Ethernet protocol communication. This low-level traffic is not protected by ISA Server. To help secure that traffic, we strongly recommend that you place a Layer-3 router between the Internet and the NLB-enabled array. This Layer-3 router will not allow the low-level Ethernet protocol to pass, thereby helping protect the array from potentially malicious Ethernet traffic from the Internet that could disrupt the operation of NLB.
• When you change the friendly name of an NLB-enabled network adapter, you must restart the Microsoft Firewall service.
• When you enable NLB, place a router in front of the NLB-enabled array. Configure the router so that it blocks raw IP traffic. Otherwise, all the array members will handle the traffic simultaneously.

Additional Resources

• Troubleshooting Network Load Balancing in ISA Server 2004 Enterprise Edition at the Microsoft TechNet Web site.
• RemoveAllNLBSettings Tool for Internet Security and Acceleration (ISA) Server 2004 Enterprise Edition at the Microsoft Download Center.
• Caching and CARP for Internet Security and Acceleration (ISA) Server 2006 at the Microsoft TechNet Web site.