Add MPI Tasks

  • Article

Applies To: Windows Compute Cluster Server 2003

If Microsoft MPI Service is to be used by an application, the application must run within an executable file called mpiexec. Except for certain development and administrative purposes, mpiexec should always be run within a job.

The executable file mpiexec is a mechanism to start a parallel MPI application. Many of the mpiexec switches and options are part of MPI standard. Microsoft MPI Service provides an mpiexec command line to support additional options. The mpiexec command line switches are based on ANL mpiexec command line.

Syntax

The syntax for mpiexec is:

mpiexec [mpiexec_options] <myapp.exe> [arguments]

In Windows Compute Cluster Server 2003, a call to mpiexec will always be submitted to Job Scheduler in a form that is equivalent to:

job submit /numprocessors:<number>[job_options] [subset_task_options] mpiexec [mpiexec_options] <myapp.exe> [arguments]

Note

/numprocessors:<number> must always be specified because the default is one processor, which results in serial execution.

[mpiexec_options] represents options entered specifically for mpiexec by the user. In practice this is rarely necessary, because the mpiexec options are largely handled by Job Scheduler through the standard job and task options. User-entered mpiexec options are supported, however, and may be used where job and task options do not have the necessary specificity. In the context of a job, mpiexec options function strictly as a subset of any equivalent job and task options. For example, mpiexec cannot specify as -hosts any hosts that have not been requested by and reserved for the job. Similarly, mpiexec cannot have a -timeout value that exceeds the job run time. An exception is the mpiexec option -n. Using this option will cause all of the processes to be run on the same processor, regardless of how many processors have been specified using /numprocessors:.

Important

Do not use the mpiexec -n option, with or without using /numprocessors:. This will always force all processes to run on a single computer.

For information about the Microsoft Message Passing Interface (MPI), see the Microsoft Web site (https://go.microsoft.com/fwlink/?LinkId=55873).

mpiexec switches

The mpiexec command has two classes of switches, or options:

  • Options that are part of Argonne National Laboratory's MPICH2 are supported by Job Scheduler.

  • Options that extend the MPICH2 distribution for the compute cluster.

    Note

    In order to avoid incompatibilities with Job Scheduler or the MPI Service, any MPICH2 options not listed in these tables are not supported.

Standard Argonne MPICH2 Options

Option Definition

-n <n>

Starts <n> number of processes.

If mpiexec is run at the command line of a compute node, and no other nodes are specified by using the –host, -hosts, or -machinefile commands, all <n> processes will be run on that node under a local instance of the MS MPI Service.

-np <n>

Starts <n> number of processes. (Same as –n option. Supported for backward compatibility with MPICH1.)

-host <hostname>

Runs processes on the specified host.

-hosts <numhosts> <host1> ... <hostn>

Runs processes on the specified hosts.

-hosts <numhosts> <host1 num_proc> ... <hostn num_proc> 

Runs specified number of processes on each specified host.

-machinefile <file>

Starts on the hosts listed in a text file. For computer file syntax, see The Message Passing Interface (MPI) Standard (https://go.microsoft.com/fwlink/?LinkId=51266).

-dir <drive:\working\directory>

Starts processes in the specified directory.

-wdir <drive:\working\directory>

Starts processes in the specified directory.

-env <variable> <value>

Sets specified environment to specified value before starting the processes. Multiple variable-value pairs can be specified. The environment variable MPICH_NETMASK <subnet_ip>/<subnet_mask> must be used in clusters with high-speed interconnect among the compute nodes but not from the head node. For example,

MPICH_NETMASK 157.59.0.0/255.255.0.0

-exitcodes

Prints process exit codes when each process exits.

-priority <class>

Sets the process priority class.

Priority classes are:

0 = idle

1 = below normal

2 = normal

3 = above normal

4 = high

The default class is 2.

port <port>

p <port>

Specifies the port that the MS MPI Service is listening on.

-path<search_path>

Specifies the search path for the user executable file.

-timeout <seconds>

Time-out for the job.

-configfile <filename> 

Configuration file from which options are to be read. Each line in the configuration file is a complete set of mpiexec options, including the user executable file and its arguments.

-genvlist <var1, var2,…varN>

Passes current values of these environment variables.

-g <local arg name>

Global version of local options: genv, gwdir, ghost, gpath, gmap.

-help, -?

Displays help for mpiexec.

-help2

Displays extended help for mpiexec.

-l

Prepends the output to the standard output file and standard error file with the rank number of the process that produced the output.

-d <flags>

Print debug output to stderr.

Flags are the sum of:

1 <error>

2 <bedug>

3 <error> + <debug>

4 <trace>

Flags are optional.

Default flag value is 3: <error> + <debug>

-job <job_context_value>

Specifies the CCP task context. mpiexec usually takes the task context from the CCP_TASKCONTEXT environment variable which is set by the Job Scheduler or node manager.