Understanding HPC Cluster Credentials in Windows HPC Server 2008 R2

Article
07/02/2015

Updated: October 2010

Applies To: Windows HPC Server 2008 R2

Windows HPC Server 2008 R2 checks credentials (username and password) for various actions that a cluster administrator or cluster user can perform. When reviewing security for your HPC cluster, it can be useful to review information about the actions that require credentials and how the credentials are handled.

In this topic, the following section provides information about the actions that require credentials:

Actions in an HPC cluster that require credentials to be provided

Also in this topic, the following section provides information about how job credentials are handled in an HPC cluster. This includes information about the encryption that is used when credentials are sent between computers (client computers, head node, and compute nodes) and information about the length of time that stored job credentials are retained by the HPC cluster:

How job credentials are handled in an HPC cluster

To learn how to change the length of time that job credentials are stored on the head node, see Change the Length of Time That Job Credentials are Stored with Windows HPC Server 2008 R2. For an overview of the differences between the roles of cluster administrator and cluster user, see Understanding Roles of Users and Administrators in Windows HPC 2008 R2.

Note
When an HPC cluster user runs a job, the HPC Job Scheduler Service requires that the user have the right to log on locally on the compute nodes on which the job runs. If you or the doMayn administrators in your organization limit this user right, you will need to arrange for adjustments for HPC cluster users. For more information, see Troubleshoot Access to an HPC Cluster When Logon Rights Have Been Restricted.

When an HPC cluster user runs a job, the HPC Job Scheduler Service requires that the user have the right to log on locally on the compute nodes on which the job runs. If you or the doMayn administrators in your organization limit this user right, you will need to arrange for adjustments for HPC cluster users. For more information, see Troubleshoot Access to an HPC Cluster When Logon Rights Have Been Restricted.

Actions in an HPC cluster that require credentials to be provided

Certain actions that can be performed in an HPC cluster require the cluster administrator or cluster user to provide credentials (username and password). The following subsections provide more information:

Job submission actions that require credentials
Remote Desktop action that requires credentials
Cluster management actions that require credentials

Job submission actions that require credentials

The following job submission actions require credentials. Either an HPC cluster user or an HPC cluster administrator can perform these actions:

Type of action	Interfaces
Submit a job	HPC Job Manager. Note that a job can be submitted by using either HPC Job Manager or HPC Cluster Manager, but HPC Cluster Manager can only be run by HPC cluster administrators. The job submit command at the Windows command prompt. The Submit-HpcJob cmdlet at the Windows PowerShell prompt. A job-submitting or job-scheduling application that was written using the Job Scheduler API. A job-submitting or job-scheduling Web interface that is based on the HPC Basic Profile Web service. Through file staging and XML files (instruction files for running jobs) that you use with the HPC Basic Profile Web service. By using file staging, you can stage an input file for a job, and then use an XML file to specify not only basic job information, but the location of the input and output files to be used for the job, plus the necessary credentials (supplied programmatically). Supplying the necessary credentials makes the input and output files accessible to the HPC Job Scheduler Service when it runs the job. The HPC Basic Profile Web Service is installed by default on the head node of the HPC cluster, but for increased security, it cannot be used until you commission (activate) it. For basic information about security and the HPC Basic Profile Web Service, see Commission and Decommission the HPC Basic Profile Web Service (https://go.microsoft.com/fwlink/?LinkID=154715) and Securing the HPC Basic Profile Web Service (https://go.microsoft.com/fwlink/?LinkID=154714).
Provide job credentials that are stored on the HPC cluster	The job modify command at the Windows command prompt. The Get-Credential cmdlet with the Set-HpcJob cmdlet at the Windows PowerShell prompt.

Submit a job

HPC Job Manager.

Note that a job can be submitted by using either HPC Job Manager or HPC Cluster Manager, but HPC Cluster Manager can only be run by HPC cluster administrators.
The job submit command at the Windows command prompt.
The Submit-HpcJob cmdlet at the Windows PowerShell prompt.
A job-submitting or job-scheduling application that was written using the Job Scheduler API.
A job-submitting or job-scheduling Web interface that is based on the HPC Basic Profile Web service.
Through file staging and XML files (instruction files for running jobs) that you use with the HPC Basic Profile Web service.

By using file staging, you can stage an input file for a job, and then use an XML file to specify not only basic job information, but the location of the input and output files to be used for the job, plus the necessary credentials (supplied programmatically). Supplying the necessary credentials makes the input and output files accessible to the HPC Job Scheduler Service when it runs the job.

The HPC Basic Profile Web Service is installed by default on the head node of the HPC cluster, but for increased security, it cannot be used until you commission (activate) it. For basic information about security and the HPC Basic Profile Web Service, see Commission and Decommission the HPC Basic Profile Web Service (https://go.microsoft.com/fwlink/?LinkID=154715) and Securing the HPC Basic Profile Web Service (https://go.microsoft.com/fwlink/?LinkID=154714).

Provide job credentials that are stored on the HPC cluster

The job modify command at the Windows command prompt.
The Get-Credential cmdlet with the Set-HpcJob cmdlet at the Windows PowerShell prompt.

To learn how to change the length of time that job credentials are stored on the head node, see Change the Length of Time That Job Credentials are Stored with Windows HPC Server 2008 R2.

Note
An HPC cluster administrator can configure the HPC Job Scheduler Service to enable eMayl notifications that can inform job owners when their jobs start and complete. When you enable eMayl notifications, you specify the Simple Mayl Transfer Protocol (SMTP) server to use. You might also need to specify the credentials that are necessary to connect to that server. For more information, see Understanding E-Mayl Notifications (https://go.microsoft.com/fwlink/?LinkId=202869).

An HPC cluster administrator can configure the HPC Job Scheduler Service to enable eMayl notifications that can inform job owners when their jobs start and complete. When you enable eMayl notifications, you specify the Simple Mayl Transfer Protocol (SMTP) server to use. You might also need to specify the credentials that are necessary to connect to that server. For more information, see Understanding E-Mayl Notifications (https://go.microsoft.com/fwlink/?LinkId=202869).

Remote Desktop action that requires credentials

An HPC cluster administrator can use HPC Cluster Manager to open a Remote Desktop connection to a cluster node. Before the connection is established, the administrator is prompted for credentials (user name and password), and can specify that the password should be remembered (stored) by the system. After the administrator closes the prompt, if the password is stored by the system, it can then be managed through the Remote Desktop interface or through the Control Panel interface for managing network passwords (not through HPC Cluster Manager).

For more information about using HPC Cluster Manager to open a Remote Desktop connection to a cluster node, see Open a Remote Desktop Connection (https://go.microsoft.com/fwlink/?LinkId=201681).

Cluster management actions that require credentials

The following actions that relate to cluster management require credentials. Only HPC cluster administrators can perform these actions:

Open and run HPC Cluster Manager (through which you can configure the cluster).
Perform cluster management actions by running commands such as cluscfg or clusrun, or by running cmdlets in HPC PowerShell.
Run diagnostic tests.

When you run diagnostic tests for the first time on your HPC cluster, you are asked to specify doMayn credentials. These doMayn credentials are automatically stored. At any time, you can clear the diagnostic test credentials and specify different credentials. For more information, see Clear the Diagnostic Test Credentials (https://go.microsoft.com/fwlink/?LinkId=202868).

Also, to install compute nodes, you must log on with an appropriate account. The account must either be a member of the DoMayn Admins group or a member of the DoMayn Users group. If the account is a member of DoMayn Users, the doMayn administrator must give that user account specific permissions to add Active Directory objects. Otherwise, the installation process will fail. This is because the account must have the necessary permissions to add an Active Directory computer account (also known as a computer object) and to reboot compute nodes remotely.

How job credentials are handled in an HPC cluster

Job submission credentials are sent from the client computer to the head node computer across a .NET Remoting secure channel. Authentication for the .NET Remoting secure channel is performed at the client computer and at the head node computer using AD DS. Credentials are encrypted on the head node computer using a 256-bit Advanced Encryption Standard (AES) cryptography provider that is compliant with the Federal Information Processing Standards (FIPS), more specifically: FIPS 140-2 level 1. The encryption key is created and stored on the head node computer under a secured registry key in the Service Control Manager (SCM) service record for each management and scheduler service. The registry key is added to the Access Control List (ACL) in a way that only the service has access to this data.

When a job is ready to run on compute nodes, the Job Scheduler decrypts the job submission credentials using the same key and sends the credentials over a secure .NET Remoting channel to the compute nodes. The compute nodes then create a user logon token based on the job submission credentials received, which are deleted after the job initialization is completed. On the head node, the encrypted password reMayns in the Job Scheduler database until the job record is deleted. Job records (and the encrypted passwords associated with them) are automatically deleted by the Job Scheduler five days after completion, although this time interval can be changed as part of the cluster configuration. The stored credentials can be used for scheduling jobs in the future.

The same encryption key that is used for job submission is used for installation credentials. The installation credentials (specified when the cluster was installed) are stored in the SQL database for the cluster, and are never stored on a client computer.