Interconnects
Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2
Q. What types of networks are required for cluster interconnects?
A. The network interface controllers along with any other components used in certified cluster configurations must have the Windows logo and appear on the Microsoft Hardware Compatibility List. The interconnect itself must support TCP/IP and UDP traffic and must appear as a single, non-routed LAN segment or subnet.
Q. Do Server clusters support high-bandwidth, low latency interconnects through WinSock direct?
A. The Server cluster code itself does not use WinSock direct path for intra-cluster communication. All communication between cluster nodes is over TCP/IP or UDP. Server clusters can use, but may not take advantage of any high-bandwidth, low latency connection between nodes that looks like a standard NDIS network adapter.
Q. How many network adapters should a cluster server have?
A. The cluster nodes must be connected by two or more independent networks in order to avoid a single point of failure. The use of two local area networks (LANs) is required. Cluster configurations that have only one network are not supported. At least 2 networks must be configured for cluster communications. Typically, one network is a private network, configured for cluster communications only and the other is a public network, configured for both client access and cluster communications.
Q. Can a cluster support multiple adapters configured as public?
A. Yes, however, there are a couple of caveats:
Each network adapter must be on a different subnet.
You can create different IP address resources that can be associated with different adapters; however, there is no way to make a resource or application dependent on the two networks so that if one fails, the application will continue to be available. In Server clusters, the dependencies must all be online for the dependent application to be online.
For future versions of the cluster technology, we are working on a feature to allow more flexible dependencies such as: a resource can depend on A OR B for it to be online.
Q. Can a cluster support multiple private networks?
A. Yes.
Q. Why do I see heartbeat packets on the networks marked as client-only?
A. In Windows 2000 and beyond, the Server cluster software heartbeats through the public network as well as the private network regardless of the configuration of the public network. This is to ensure that the cluster service can detect failures of the public network adapters and can failover an application if the node currently hosting it cannot communicate with the outside world.
Q. How should the different networks be configured?
A. The cluster nodes must be connected by two or more independent networks in order to avoid a single point of failure. The use of two local area networks (LANs) is typical. The configuration of a cluster with nodes connected by only one network is not supported.
You should configure the private network as Internal Cluster Communication Only and the public network as All Communications.
Q. What type of information is carried over the cluster network?
A. The cluster service uses the cluster network for the following intra-cluster traffic:
Server cluster query/management and control information
Heartbeats for failure detection
Intra-cluster communication to ensure tight consistency of cluster configuration
Cluster join requests when a node reboots or recovers from a failure
Q. What types of protocol are supported for applications in a Server cluster?
A. Cluster-aware applications must use protocols based on top of TCP/IP. The cluster software only supports the TCP/IP protocols for failover out-of -the-box.
Q. Will failover occur if the public network on one server fails?
A. Yes, in Windows 2000 and above, additional heartbeats are sent across the public networks to detect public network and/or network adapter failures and to cause applications to failover if the node they are currently hosted on cannot communicate with the other nodes.
Q. Is network adapter teaming supported in a Server cluster?
A. Yes, however there are caveats. The use of network adapter teaming on all cluster networks concurrently is not supported. At least one of the cluster networks that are enabled for internal communication between cluster nodes must not be teamed. Typically, the un-teamed network is a private interconnect dedicated to this type of communication. The use of network adapter teaming on other cluster networks is acceptable; however, if communication problems occur on a teamed network, Microsoft Product Support Services may require that teaming be disabled. If this action resolves the problem or issue, then you must seek further assistance from the manufacturer of the teaming solution.
Q. Is DHCP supported for Server cluster virtual servers?
A. No, virtual servers must have static IP addresses.
Q. Is DHCP supported for Server cluster nodes?
A. Yes, addresses may be assigned to the physical nodes dynamically by DHCP, but manual configuration with static addresses is recommended.
Q. Is NetBIOS required to run Server clusters?
A. In Windows NT and Windows 2000, NetBIOS is required for Server clusters to function correctly.
In Windows Server 2003, the cluster service does not require NetBIOS. A basic principle of server security is to disable unneeded services. To determine whether to disable NetBIOS, consider the following:
Some services and applications other than the cluster service use NetBIOS. Review all the ways a clustered server functions before disabling NetBIOS on it.
With NetBIOS disabled, you will not be able to use the Browse function in Cluster Administrator when opening a connection to a cluster. Cluster Administrator uses NetBIOS to enumerate all clusters in a domain.
With NetBIOS disabled, Cluster Administrator does not work if a cluster name is specified.
You can control the NetBIOS setting for the cluster through the Cluster IP Address resource property sheet (parameters).
Q. Is IPSec supported in a Server cluster?
A. Although it is possible to use Internet Protocol Security (IPSec) for applications that can failover in a server cluster, IPSec was not designed for failover situations and we recommend that you do NOT use IPSec for applications in a server cluster.
Q. How do Server clusters update router tables when doing IP failover?
A. As part of its automatic recovery procedures, the cluster service will issue IETF standard ARP "flush" commands to routers to flush the machine addresses related to IP addresses that are being moved to a different server.
Q. How does the Address Resolution Protocol (ARP) cause systems on a LAN to update their tables that translate IP addresses to physical machine (MAC) addresses?
A. The ARP specification states that all systems receiving an ARP request must update their physical address mapping for the source of the request. The source IP address and physical network address are contained in the request. As part of the IP address registration process, the Windows TCP/IP driver broadcasts an ARP request on the appropriate LAN several times. This request asks the owner of the specified IP address to respond with its physical network address. By issuing a request for the IP address being registered, Windows can detect IP address conflicts; if a response is received, the address cannot be safely used. When it issues this request, though, Windows specifies the IP address being registered as the source of the request. Thus, all systems on the network will update their ARP cache entries for the specified address, and the registering system becomes the new owner of the address.
Note that if an address conflict does occur, the responding system can send out another ARP request for the same address, forcing the other systems on the subnet to update their caches again. Windows does this when it detects a conflict with an address that it has successfully registered.
Q. Server clusters use ARP broadcasts to re-set MAC addresses, but ARP broadcasts don't pass routers. So what about clients behind the routers?
A. If the clients were behind routers, they would be using the router(s) to access the subnet where the cluster servers were located. Accordingly, the clients would use their router (gateway) to pass the packets to the routers through whatever route (OSPF, RIP, etc) is designated. The end result is that their packet is forwarded to a router on the same subnet as the cluster. This router's ARP cache is consistent with the MAC address(es) that have been modified during a failover. Packets thereby get to the correct Virtual server, without the remote clients ever having seen the original ARP broadcast.