Best Practices for Optimizing Job Completion Time

In Microsoft HPC Pack, a job is a request for cluster resources. How long a job waits in the queue depends on a number of factors, including:

  • What resources the job needs

  • What jobs are currently running

  • What other jobs are in the queue

  • The scheduling policies configured by the cluster administrator

Job properties define how a job and its tasks run. When setting values for job properties, use the following guidelines to decrease the amount of time your job must wait in the queue:

  • The higher the priority of a job, the sooner it will run.

  • Always specify a run time and be as accurate as possible (without underestimating the run time). Jobs with an accurate run time specified are most likely to take advantage of the Backfill scheduling policy (if enabled).

  • Set the minimum resource requirement for your job as low as possible.

  • Be unspecific about the types of resources required. Jobs that require specific resources (or specific types of resources) may need to wait for those specific resources even when other resources are available.

Additional references