Skip to main content
Steps: Submit a Job with a Node Preparation and a Node Release Task

Updated: March 30, 2011

Applies To: Windows HPC Server 2008 R2

This topic walks through an example of how to submit a job to a Windows HPC Server 2008 R2 cluster. The job copies an application to each node that is allocated to the job, runs the application against a set of 1,000 indexed files, and then cleans up the nodes when the job completes. The sample job includes a Node Preparation task, a Parametric Sweep task, and a Node Release task. For more information, see Understanding Task Types.

The procedures in this topic describe three methods to create and submit the sample job. To understand the commands in the procedures, review the information in Before you start.

In this topic:

Before you start

To understand the tasks and commands that are specified in the sample job, review the information in this section.

How the sample job works

The sample job runs as follows:

  • The Node Preparation task runs on each node that is allocated to the job. The task copies the application from a shared folder on the network to a local folder on each node that is allocated to the job.

  • Each subtask in the Parametric Sweep task runs an instance of the application against one of 1,000 indexed files that are stored in a shared folder on the network.

  • The Node Release task runs on each node as it is released from the job. The task deletes the application and any temporary files that the application produced from each allocated node.

The procedures uses placeholder names to specify the name of the application and the shared folders and files for the job. The following list describes the files and applications for the sample job:

  • The application is named myApp.exe.

  • myApp.exe is stored on a shared folder on the network that is named \\networkShare\bin.

  • myApp.exe accepts a single file as input.

  • The files that we want to pass to myApp.exe are a series of 1,000 indexed files that are named in0001.txt through in1000.txt.

  • The input files are stored on a shared folder on the network that is named \\datashare\userName.

  • myApp.exe creates temporary files in the working directory that are in the form myApp.temp.xxxxxxx, where xxxxxxx can be any string.

What you need to know about the asterisk (*)

In tasks that include sub-tasks, you can use the asterisk (*) character as a placeholder for the parametric sweep index (in Parametric Sweep tasks) or for the sub-task ID (in Service, Node Preparation, and Node Release tasks). For example, in a parametric sweep task with 1000 sub-tasks, the job scheduler interprets in*.txt as in1.txt in the first sub-task, and as in1000.txt in the last sub-task.

You can include more than one asterisk to indicate the minimum number of positions to use when expressing the sweep index or sub-task number. In the sample job, the indexed input files are named in0001.txt through in1000.txt, so the sample procedures specify the sweep index from 1-1000, and the input files in the form in****.txt.

In tasks that include sub-tasks, the job scheduler interprets the asterisk (*) as a placeholder before passing the command to the compute nodes. To run a command that uses an asterisk as a wildcard, include the caret (^) as an escape character. In the sample job, the Node Release task includes a command to delete all the files that start with “myApp.temp.” from the node. The command must include a caret as follows:

del myApp.tmp.^*

Without the caret, the command for the first instance of the node release task attempts to delete a file named myApp.tmp.1 from the compute node. With the caret, the command attempts to delete all files that start with “myApp.temp.” from the node (myApp.temp.*).

Commands that are submitted from a command prompt window are additionally interpreted before they are passed to the job scheduler. For more information, see the Submit the sample job by using a command prompt window section in this topic.

^ Top of page

Submit the sample job by using a command prompt window

You can use the job command to create and submit jobs to a Windows HPC cluster from a command prompt window. In the following example, the ID of the new job is 42.

job new
job add 42 /type:nodeprep copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin

job add 42 /parametric:1-1000 /workdir:c:\Users\userName\bin myApp.exe \\datashare\userName\in****.txt

job add 42 /type:noderelease /workdir:c:\Users\userName\bin del /Q myApp.exe ^& del /Q myApp.tmp.^^^*

Commands that are submitted from a command prompt window are interpreted before they are passed to the job scheduler. The job scheduler also interprets commands before passing them to the compute nodes. You can use one or more carets (^) as escape characters to control when special characters are interpreted.

In the sample node release task, note the following uses of the caret (^):

  • The ^& tells the command prompt not to interpret the & character. Without the caret, the command attempts to delete a file named myApp.tmp.^* from the local computer, and also adds a task to job 42 that only deletes myApp.exe. With the caret, the command adds a task to job 42 that deletes the application and the temporary files.

  • The ^^^* tells the command prompt not to interpret the second caret and not to interpret the asterisk (*). The command prompt passes ^* to the job scheduler. The caret in front of the asterisk is necessary so that the job scheduler, in turn, does not interpret the asterisk as a placeholder for the sub-task ID.

To monitor the progress of your job, see Verification: Check Job Completion and Output.

^ Top of page

Submit the sample job by using HPC PowerShell

You can use the New-HpcJob, Add-HpcTask, and Submit-HpcJob cmdlets to create and submit jobs to a Windows HPC cluster by using HPC PowerShell. To open HPC PowerShell, click Start, point to All Programs, click Microsoft HPC Pack 2008 R2, and then click HPC PowerShell.

In the following example, the variable $j stores the new job ID so that you do not have to specify the job ID for each cmdlet.

$j=New-HpcJob

Add-HpcTask –job $j –Type NodePrep –Command "copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin"

Add-HpcTask –job $j –Type ParametricSweep –start 1 –end 1000 –WorkDir c:\Users\userName\bin –command "myApp.exe \\datashare\userName\in****.txt"

Add-HpcTask –job $j –Type NodeRelease –WorkDir c:\Users\userName\bin –Command "del /Q myApp.exe & del /Q myApp.tmp.^*"

Submit-HpcJob –id $j

noteNote
In the Node Release task, use the caret (^) as an escape character in front of the asterisk (*) so that the job scheduler does not interpret the asterisk as a placeholder for the sub-task ID.

To monitor the progress of your job, see Verification: Check Job Completion and Output.

^ Top of page

Submit the sample job by using HPC Job Manager

To open HPC Job Manager, click Start, point to All Programs, click Microsoft HPC Pack 2008 R2, and then click HPC Job Manager.

To create and submit the sample job
  1. In HPC Job Manager, in the Actions pane, click New Job.

  2. In the left pane of the New Job dialog box, click Edit Tasks.

  3. Add the Node Preparation task:
    1. Point to the Add button, click the down arrow, and then click Node Preparation Task.

    2. In the task dialog box, in Command line, type the following command:

      copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin

    3. Click OK to add the task to your job. The new task now appears in the list of tasks for your job.

  4. Add the Parametric Sweep task:
    1. Point to the Add button, click the down arrow, and then click Parametric Sweep Task.

    2. In the task dialog box, specify the number of steps for the sweep task by setting the Start value to 1 and set the End value to 1000.

    3. In Command line, type the following command (relative to the working directory):

      myApp.exe \\datashare\userName\in****.txt

    4. In Working directory, type the following:

      c:\Users\userName\bin

    5. Verify your index series under Step 4. Expand the Command Line column to see the full commands for each subtask. For this task, the preview pane displays the following:

      Task.exe myApp.exe \\datashare\userName\in0001.txt Task.exe myApp.exe \\datashare\userName\in0002.txt…Task.exe myApp.exe \\datashare\userName\in1000.txt
    6. Click OK to add the task to your job.

  5. Add the Node Release task:
    1. Point to the Add button, click the down arrow, then click Node Release Task.

    2. In the task dialog box, in Command line, type the following command:

      del /Q myApp.exe & del /Q myApp.tmp.^*

      noteNote
      Use the caret (^) as an escape character in front of the asterisk (*) so that the job scheduler does not interpret the asterisk as a placeholder for the sub-task ID.

    3. In Working directory, type the following:

      c:\Users\userName\bin

    4. Click OK to add the task to your job.

  6. Click Submit.

To monitor the progress of your job, see Verification: Check Job Completion and Output.

^ Top of page

Submitting the sample job by using the scheduler APIs

The following C# code snip illustrates how to create and submit the sample job by using the Microsoft.Hpc.Scheduler APIs.

try {
                using (IScheduler scheduler = new Scheduler()) {
                    try {
                        scheduler.Connect(hostname);
                    } catch (Exception e) {
                        Console.Error.WriteLine("Unable to connect to " + hostname);
                        Console.Error.WriteLine(e.Message);
                        return -1;
                    }
                    ISchedulerJob myjob = scheduler.CreateJob();
                    myjob.AutoCalculateMax = true;
                    myjob.AutoCalculateMin = true;

                    // Node Prep task
                    ISchedulerTask task = myjob.CreateTask();
                    task.Type = TaskType.NodePrep;
                    task.CommandLine = "copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin";
                    myjob.AddTask(task);

                    // Parametric task
                    ISchedulerTask task2 = myjob.CreateTask();
                    task2.Type = TaskType.ParametricSweep;
                    task2.WorkDirectory = "c:\Users\userName\bin";
                    task2.EndValue=1000;
                    task2.CommandLine = "myApp.exe \\datashare\userName\in****.txt";
                    myjob.AddTask(task2);

                    // Node Release task
                    ISchedulerTask task3 = myjob.CreateTask();
                    task3.Type = TaskType.NodeRelease;
                    task3.WorkDirectory = "c:\Users\userName\bin";
                    task3.CommandLine = "del /Q myApp.exe & del /Q myApp.tmp.^*";
                    myjob.AddTask(task3);



                    scheduler.SubmitJob(myjob, null, null);
                }
      } catch(Exception e1){
                    Console.WriteLine(e1.Message);
                    Console.WriteLine(e1.StackTrace);
    }

^ Top of page

Additional considerations
  • To complete this procedure, you must have been added as a user on the cluster, or you must be a member of a domain group that has been added as a user on the cluster. For more information, contact your cluster administrator.

  • To submit jobs to a cluster, you must have the HPC Pack 2008 R2 client utilities installed on your computer. To connect to a cluster, your computer must be joined to the same domain or to a domain that has a trust relationship with the cluster domain.

  • The procedures use the default job template to create the job. Cluster administrators can specify permissions regarding which users can use a particular job template. For information about the job templates that you have access to and should use, contact your cluster administrator.