Use Batch and Script Files with Jobs in Windows Compute Cluster Server 2003

  • Article

Applies To: Windows Compute Cluster Server 2003

You can run a batch file or script as the command line of a task or to generate and submit a job. Preexisting batch files and scripts can be migrated easily to Windows Compute Cluster Server 2003 from other platforms. A batch file or script must meet the following requirements to be compatible with the cluster environment:

  • Built-in commands and path format must be compatible with Windows. This is the Windows compatibility requirement.

  • Script and batch files not entered as command lines for tasks within jobs must generate commands in the format job submit myapp.exeorjob submit mpiexec myapp.exe. This is the command context requirement.

Windows compatibility requirement

Paths and built-in commands in the batch file must be compatible with Windows. For example, a template for a UNIX batch file might look like this:

:: set up environment variables
setenv Path=“~/user/bin”
setenv LM_LICENSE_FILE=“ /vendor/license”
:: Invoke application
mpiexec myapp.exe

The equivalent Windows template would look like this:

:: set up environment variables
set Path=“C:\program files\vendor\bin”
set LM_LICENSE_FILE=“c:\program files\vendor\license.bat”
:: Invoke application
mpiexec myapp.exe

Command context requirement

Batch files and scripts that run jobs and tasks are either:

  • Standalone (run from the Windows command prompt)

  • Task command line

Standalone scripts

Batch files and scripts that are not entered at the Windows Command prompt must create and submit the jobs themselves. An example is the following Perl script, which creates a parametric sweep:

#Create new job
`job new /numprocessors:8`;
#Add parametric tasks
for ($i = 0; $i < 50; $i++) {
    `job add %CCP_JOBID% /stdin:\\fileshare\infile$i.txt /stdout:outfile$i.txt myapp.exe`;
}
`job submit /ID:%CCP_JOBID%`;

This script creates 50 parametric tasks, each using a separate remote input file and writing its results to a separate remote output file.

For jobs like this, it is usually more useful to read and write to local data files. This can be done as follows:

  1. Copy the input files from the remote file server to your local working directory before the parametric tasks begin.

  2. Copy the output files to the file server when the parametric tasks are complete.

Although a standalone script could include these staging tasks, this method can present a disadvantage. Stage-in and stage-out tasks are serial and must be targeted to individual nodes. However, unless your job needs to reserve all nodes in the cluster to meet its minimum processors requirement, or unless you have specified certain nodes in advance, it is not predictable which nodes will be reserved for the job. This information is contained in the environment variable CCP_NODES, but CCP_NODES is meaningful only in the context of a task. For this reason, scripts that include staging are best submitted as task command lines. For more information, see Recursive task command-line scripts.

Task command-line scripts

Unlike standalone scripts, scripts entered as task command lines within jobs may call virtually any command, including any CLI command. The only restrictions are the same restrictions that apply to any command entered as a job task:

  1. MPI commands must be called in the format mpiexec myapp.exe to run in parallel.

  2. All tasks will run in batch mode, with no screen input or output.

Non-recursive task command-line scripts

The simplest task command-line scripts run multiple commands in Windows in a single task. Example:

for ($i = 1; $i < 100001; $i++) {
    `myapp.exe $i output outfile$i`;
}

This type of script is useful when the number of command instances is very large and each has a very short runtime. In this case the startup overhead of running each command as a separate task can outweigh the advantages of fine-grained distribution across the multiple processors. Such tasks must be used with caution. Because they all run within a single task, they will not run in parallel with one another as they would as separate tasks. As a result, the following general rule applies:

To parallelize a job consisting of multi-command tasks in script form, divide the command instances into at least as many command scripts as there are available processors. For example:

script1.pl:

for ($i = 1; $i < 25001; $i++) {
    `myapp.exe $i output outfile$i;
}

script2.pl: for ($i = 25001; $i < 50001; $i++) {

    `myapp.exe $i output outfile$i;
}

script3.pl:

for ($i = 50001; $i < 75001; $i++) {
    `myapp.exe $i output outfile$i; }
}

script4.pl:

for ($i = 75001; $i < 100001; $i++) {
    `myapp.exe $i output outfile$i;
}

Then submit the job as follows:

job new /numprocessors:4

job add <job_id> perl script1.pl

job add <job_id> perl script2.pl

job add <job_id> perl script3.pl

job add <job_id> perl script4.pl

job submit /ID:<job_id>`;

Recursive task command-line scripts

Because CLI commands are allowed as task commands, you can run job new, job submit, and job add as tasks, allowing you to add tasks within a task or even create a new job.

This permits you to take advantage of built-in environment variables, like CCP_NODES, that are not set until the job is submitted and activated. The following Perl script shows how the parametric sweep (shown in the previous section) can be run in a script that includes staging in and out of data files.

# Script is submitted in a command like this:
# job submit /numprocessors:8 perl staged_sweep.pl

#Stage input files in

$path1 = "\\\\server1\\hpc_files\\infile*.txt";

# Split env var CCP_NODES into a space-delimited array @array, then
# push every second value (which is a node name) into array @nodes.

@array = split (/\s/, $ENV{CCP_NODES});
shift(@array);
while (@array) {
push @nodes,shift(@array);
shift(@array);
}

# Stage input files in

for(@nodes) {

`job add %CCP_JOBID% /name:task1 /requirednodes:$_ xcopy /d /y $path1 .`;
}

# Add parametric tasks

for ($i = 1; $i < 11; $i++) {
    `job add %CCP_JOBID% /name:task2 /depend:task1 /stdin:infile$i.txt /stdout:outfile$i.txt myapp.exe`;
}

# Stage output files out
$path2 = "\\\\server1\\hpc_files\\";

for (@nodes)  {

`job add %CCP_JOBID% /name:task3 /depend:task2 /requirednodes:$_ xcopy /d /y outfile*.txt $path2`;
}

#Clean out working directories

for (@nodes) {
`job add %CCP_JOBID% /name:task4 /depend:task3 /requirednodes:$_ erase /f
infile*.txt`;
`job add %CCP_JOBID% /name:task4 /depend:task3 /requirednodes:$_ erase /f
outfile*.txt`;
}

Using the Compute Cluster Pack API (CCPAPI)

The previous example scripts call CCS command line interface commands. This is done for greater ease of scripting and because it does not involve learning a new language. In some cases, jobs will run more efficiently if you bypass the CLI and instead access the underlying Compute Cluster Pack API (CCPAPI) directly, which is the API for the Job Scheduler (and for the Cluster Administrator as well).

For more information about creating jobs using the CCPAPI, see Compute Cluster Server Command Line Interface Reference (https://go.microsoft.com/fwlink/?LinkID=99365).

For links to MSDN documentation about CCPAPI, see Creating a Custom User Interface.