Understanding Security Considerations for Network Topologies in Windows HPC Server 2008 R2
Updated: October 2010
Applies To: Windows HPC Server 2008 R2
In a cluster running Windows HPC Server 2008 R2, the network topology is the organization of the physical networks that connect the head node and the compute nodes. There are five network topologies that you can choose from when setting up the physical networks. (After setting up the networks, as you run the Network Configuration Wizard on the head node, you must specify the topology that you chose.)
.gif) Important |
|---|
| The network topology in an HPC cluster is determined by the physical organization of the networks, not just by the software settings. You must make the choice of a network topology during the design phase, not after the cluster is running and in use. |
For a review of the network topologies to choose from in an HPC cluster, see Categories of network topologies in HPC clusters, later in this topic.
Choosing network topology 1 or 3 isolates the compute nodes on one or more separate networks, which can increase the level of security. For more information, see Considerations for choosing network topology 1 or 3 for an HPC cluster, later in this topic.
For a cluster that will run Message Passing Interface (MPI) jobs, we recommend network topology 1, 2, 3, or 4. Choosing one of these topologies ensures that MPI messages can be passed on a private or application network, rather than having to use the enterprise network (which is the only network available with network topology 5). For more information about this recommendation, see Considerations for an HPC cluster that will run MPI jobs, later in this topic.
Considerations for choosing network topology 1 or 3 for an HPC cluster
When you choose a network topology for an HPC cluster, the Mayn security consideration to keep in mind is that network topologies 1 and 3 isolate the compute nodes on one or more separate networks. In topology 1, compute nodes are isolated on a private network, and in topology 3, compute nodes are isolated on private and application networks. Because the compute nodes are isolated, these topologies can strengthen the security of an HPC cluster.
For more information about other implications of choosing a network topology, such as implications related to performance or the development and debugging of parallel applications for use on an HPC cluster, see Appendix 1: HPC Cluster Networking (https://go.microsoft.com/fwlink/?LinkId=198313) in the Design and Deployment Guide for Windows HPC Server 2008 R2.
Considerations for an HPC cluster that will run MPI jobs
For an HPC cluster that will run Message Passing Interface (MPI) jobs, we recommend network topology 1, 2, 3, or 4. The choice of network topology is relevant with MPI jobs because (unlike other jobs) the authentication for an MPI job in the cluster happens only when the job is initiated. After that, MPI messages are passed between nodes without additional authentication or encryption. This substantially improves performance because it helps to minimize the time that is needed for passing messages between nodes. If a cluster that runs MPI jobs includes a private network, and possibly an application network also—that is, if the cluster uses network topology 1, 2, 3, or 4—MPI messages will by default be passed on the private or application network, not on the enterprise network. This can help protect the data in the messages. In contrast, with network topology 5, only the enterprise network is available, and MPI messages will therefore be passed on that network, which could make them easier for an attacker to gain access to.
.gif) Note |
|---|
| In Windows HPC Server 2008 R2, MPI messages will by default be passed on the private or application network, but the environment variable that controls this might have been changed by an administrator, and might need to be corrected. For more information, see Review or Adjust the Network That is Used for MPI Messages with Windows HPC Server 2008 R2. |
Also, for HPC clusters that run MPI jobs, we recommend that you practice basic network security by restricting physical access to your networks (especially private or application networks) and monitoring the networks to detect unauthorized computers. This is part of using defense in depth. Also, if you are using multiple networks, connect network cables carefully, to make sure that a network cable for one network is not crossed with the cable for another.
For more information about other implications of choosing a network topology, such as implications related to performance or the development and debugging of parallel applications for use on an HPC cluster, see Appendix 1: HPC Cluster Networking (https://go.microsoft.com/fwlink/?LinkId=198313) in the Design and Deployment Guide for Windows HPC Server 2008 R2.
Categories of network topologies in HPC clusters
This section lists the five network topologies that are available in clusters running Windows HPC Server 2008 R2, and categorizes them according to criteria such as whether the compute nodes are isolated. For a complete description of the five network topologies (including diagrams) and the process of preparing for a deployment, see Appendix 1: HPC Cluster Networking (https://go.microsoft.com/fwlink/?LinkId=198313) and Step 1: Prepare for Your Deployment (https://go.microsoft.com/fwlink/?LinkId=201563) in the Design and Deployment Guide for Windows HPC Server 2008 R2.
The network topologies in Windows HPC Server 2008 R2 can be categorized as follows:
Network topologies in which the compute nodes are isolated
In the following topologies, the head node is on the enterprise network, and the compute nodes are isolated on one or more separate networks. Therefore, the compute nodes have an extra layer of security, because traffic to them passes through the head node.
Topology 1: Compute Nodes Isolated on a Private Network
Topology 3: Compute Nodes Isolated on Private and Application Networks
Network topologies in which nodes are not isolated, but have access to a network separate from the enterprise network
In the following topologies, the compute nodes are not isolated, but have access to at least one network separate from the enterprise network.
Topology 2: All Nodes on Enterprise and Private Networks
Topology 4: All Nodes on Enterprise, Private, and Application Networks
Network topology in which only one network is available
In the following topology, all nodes connect to one network only.
- Topology 5: All Nodes on an Enterprise Network