Use ClusterCopy to Move Files to and from Nodes in a Windows HPC Cluster

  • Article

Applies To: Microsoft HPC Pack 2008, Microsoft HPC Pack 2008 R2, Microsoft HPC Pack 2012, Windows HPC Server 2008, Windows HPC Server 2008 R2

This topic describes how to use ClusterCopy to rapidly copy files to on-premises compute nodes on a Windows HPC cluster. You can also use ClusterCopy to help collect files from cluster nodes. ClusterCopy is included in the HPC Pack Tool Pack. For more information, see Microsoft HPC Pack: Tools and Extensibility.

This topic includes the following sections:

  • Requirements

  • How ClusterCopy works

  • Using ClusterCopy

  • ClusterCopy configuration parameters

Requirements

To use ClusterCopy, you need the following software and permissions:

  • HPC Pack client utilities installed on the client computer.

  • ClusterCopy installed on the head node or on the client computer that you are using to access the head node.

  • User permissions on the HPC cluster.

  • Read permissions to the source folder.

    Read/write permissions to the destination folder.

How ClusterCopy works

File distribution

When distributing files to cluster nodes, ClusterCopy creates a series of tasks that define the copy behavior (if you specify the UseMutlipleJobs parameter, ClusterCopy creates a series of jobs instead). The first task copies files from the source folder to a set of nodes. The number of nodes in each set is defined by the ParentSpan parameter (by default, this has a value of 3).

After the first task completes, the files exist in ParentSpan + 1 locations. Each of these locations is used as a source for the next round of copy tasks, until the files have been copied to all of the specified nodes. If files are not copied successfully to a particular node, all copy jobs or tasks that depend on that node fail. ClusterCopy creates a log file in the install directory that lists failed nodes.

The following diagram illustrates an example of how ClusterCopy distributes files to cluster nodes:

How ClusterCopy distributes files to nodes

File collection

When collecting files from cluster nodes, ClusterCopy creates a series of tasks or jobs that define the copy behavior. Each task copies files to the destination folder from the number of nodes that are specified by the ParentSpan parameter. When the first task is finished (successfully or not), the next task begins, and so on until ClusterCopy has attempted to collect files from all of the specified nodes. ClusterCopy creates a log file in the install directory that lists failed nodes.

Using ClusterCopy

You can configure and run ClusterCopy by using a Command Prompt window or the ClusterCopy dialog box. To use a Command Prompt window, run ClusterCopyConsole.exe. To use the dialog box, run ClusterCopy.exe.

The following procedures describe how to use ClusterCopy. For detailed information about the configuration parameters, see ClusterCopy configuration parameters.

Note

The default installation folder is C:\Program Files\Microsoft HPC Pack <Version> Tool Pack\ClusterCopy.

To use ClusterCopy at a command prompt

  1. Open a Command Prompt window and go to the ClusterCopy installation folder. For example, type a command similar to the following:

    cd C:\Program Files\Microsoft HPC Pack 2012 Tool Pack\ClusterCopy

  2. Run ClusterCopyConsole, and then specify the parameters for the head node, source folder, destination folder, and the compute nodes that you want files copied to or from. To collect files from the nodes, include the /Collect parameter.

    The following code samples provides examples with and without the /Collect parameter:

    • To copy all files from C:\hpc\data on the head node to C:\hpc\scratch on all nodes in the ComputeNodes and WorkstationNodes groups, type:

      clustercopyconsole /HeadNode:HEADNODE /SourceFolder:C:\hpc\data /DestinationFolder:C:\hpc\scratch /NodeGroups:ComputeNodes;WorkstationNodes

    • To collect file1.dat and file2.dat from C:\hpc\scratch on all nodes to C:\hpc\results on the head node:

      clustercopyconsole /HeadNode:HEADNODE /SourceFolder:C:\hpc\scratch /TargetFiles:file1.dat;file2.dat /DestinationFolder:C:\hpc\data /Collect

  3. When the job is complete, a message appears that indicates the number of successful and failed copies. You can read the list of failed nodes in a log file in the ClusterCopy install folder.

To use the ClusterCopy dialog box

  1. Navigate to the location where you installed ClusterCopy, and then double-click ClusterCopy.exe.

  2. Specify the head node, source folder, destination folder, and the compute nodes that you want files copied to or from.

  3. To distribute files to nodes, click Copy to Nodes (Distribute). To collect files from nodes, click Copy from Nodes (Collect).

  4. The ClusterCopy dialog box displays the progress of your copy job.

  5. When the job is complete, a message box appears that indicates the number of successful and failed copies. You can read the list of failed nodes in a log file in the ClusterCopy install folder.

The following screenshot illustrates the ClusterCopy dialog box:

Dialog box for ClusterCopy

ClusterCopy configuration parameters

The following table describes the parameters that you can use to configure ClusterCopyConsole (you can configure the same parameters in the ClusterCopy dialog box).

Purpose Parameter Description

Specify head node

/HeadNode:<Server>

(or /HN:<Server>)

Specifies the name of the head node in the HPC cluster. You must include this parameter if you are not running ClusterCopyConsole on the head node or if the CCP_SCHEDULER environment variable on your computer does not already specify the name of the head node.

Specify source files

/SourceFolder:<Folder>

(or /SF:<Folder>)

Specifies the folder where the files that you want to copy are stored. This can be a local folder on the head node or on cluster nodes, or a shared folder on the network. To copy files from a shared folder on the network, specify the UNC path for the shared folder (for example: \\server\folder).

Also consider the following default information:

  • By default, files are copied to or from all cluster nodes that are Online and reachable. To specify a subset of nodes, include the /NodeGroups or /NodeFiles parameter.

  • By default, the files from a shared folder on the network are copied to the head node first, and then to the compute nodes. To copy files directly from the network folder, include the /UseClusterFileShare parameter in your command.

  • By default, all files in the folder are copied. To copy specific files in the folder, include the /TargetFiles parameter in your command.

  • By default, files are distributed to cluster nodes. To collect files from nodes, include the /Collect parameter.

/UseClusterFileShare

(or /UCFS)

Specifies that the files can be copied directly from the shared folder on the network that you specified as the source folder. If you do not include this parameter and you specified a shared folder on the network as the source folder, the files are first copied to the head node.

To use this parameter, the shared folder on the network must be accessible to the compute nodes.

/TargetFiles:<Files>

(or /TF:<Files>)

Lists the files in the source folder that you want to copy. If you do not include this parameter, all files and subfolders in the source folder are copied.

Specify destination folder

/DestinationFolder:<Folder>

(or /DF:<Folder>)

Specifies the folder where the files will be copied. If the destination folder does not exist, ClusterCopyConsole creates the folder.

To copy files from the nodes to a shared folder on the network, specify the UNC path for the shared folder (for example: \\server\folder). All of the nodes must be able to access this folder.

Specify nodes

/NodeGroups:<Groups>

(or /NG:<Groups>)

List the node groups that you want to copy files to or from. Use semicolons (;) to specify more than one node group (for example, /NG: ComputeNodes; WorkstationNodes;…).

ClusterCopy uses the union of the listed node groups, not the intersection.

Only nodes that are online and reachable are used. If you do not specify this parameter or the /NodeFiles parameter, all nodes in the HPC cluster are used.

/NodeFiles:<Files>]

(or /NF:<Files>)

Lists the text files that include a list of nodes to use when copying files. The text files must list one node per line with no punctuation. Use semicolons (;) to specify more than one text file (for example, /NF: nodelist1.txt; nodelist2.txt; nodelist3.txt…).

Only nodes that are online and reachable are used. If you do not specify this parameter or the /NodeGroups parameter, all nodes in the HPC cluster are used.

Configure copy behavior

/Collect

(or /C)

Specifies that files are collected from the cluster nodes to the destination folder. If you do not specify this parameter, files are copied to cluster nodes.

/UseMultipleJobs

(or /UMJ)

Specifies that a separate job must be created for each copy task. If you do not specify this parameter, all copy tasks are included in one job.

When ClusterCopy runs as a single job, the job remains queued until all of the specified nodes are available. When ClusterCopy runs as multiple jobs, copying can begin before all the nodes become available.

/ParentSpan:<Targets>

(or /PS:<Targets>)

Specifies the number of simultaneous files that are copied from a source when distributing files. When collecting files, specifies the number of simultaneous files that are copied to the destination folder.

If you do not specify this parameter, the default value of 3 is used.

Considerations:

  • You can adjust this setting according to the network bandwidth of your Private network.

  • When distributing files, each node that has a copy of the files becomes a new source for the next round of copy tasks (or jobs). So even if this number is set to 1, you can still experience improved copying performance.