Authentication and Data Encryption for Windows Computers in Operations Manager 2007

Applies To: Operations Manager 2007 R2, Operations Manager 2007 SP1

Operations Manager 2007 consists of components such as the root management server, management server, gateway server, Reporting Server, Operations Manager database, Reporting data warehouse, agent, Web console, and Operations console. This section explains how authentication is performed and identifies connection channels where the data is encrypted.

Certificate-Based Authentication

When an Operations Manager agent and management server are separated by either an untrusted forest or workgroup boundary, certificate-based authentication will need to be implemented. The following sections provide information about these situations and specific procedures for obtaining and installing certificates from Windows-based certification authorities.

Setting Up Communication Between Agents and Management Servers Within the Same Trust Boundary

An agent and the management server use Windows authentication to mutually authenticate with each other before the management server accepts data from the agent. The Kerberos version 5 protocol is the default method for providing authentication. In order for Kerberos-based mutual authentication to function, the agents and management server must be installed in an Active Directory domain. If an agent and a management server are in separate domains, full trust must exist between the domains. In this scenario, after mutual authentication has taken place, the data channel between the agent and the management server is encrypted. No user intervention is required for authentication and encryption to take place.

Setting Up Communication Between Agents and Management Servers Across Trust Boundaries

An agent (or agents) might be deployed into a domain (domain B) separate from the management server (domain A), and no two-way trust might exist between the domains. Because there is no trust between the two domains, the agents in one domain cannot authenticate with the management server in the other domain using the Kerberos protocol. Mutual authentication between the Operations Manager 2007 components within each domain still occurs.

A solution to this situation is to install a gateway server in the same domain where the agents reside, and then install certificates on the gateway server and the management server to achieve mutual authentication and data encryption. The use of the gateway server means you need only one certificate in domain B and only one port through the firewall, as shown in the following illustration.

For more information, see the following topics in this security guide:

How to Obtain a Certificate Using Windows Server 2003 Enterprise CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2003 Stand-Alone CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2008 Enterprise CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2008 Stand-Alone CA in Operations Manager 2007

Setting Up Communication Across a Domain – Workgroup Boundary

In your environment, you may have one or two agents deployed to a workgroup inside your firewall. The agent in the workgroup cannot authenticate with the management server in the domain using the Kerberos protocol. A solution to this situation is to install certificates on both the computer hosting the agent and the management server that the agent connects to, as shown in the following illustration.

Perform the following steps on both the computer hosting the agent and the management server using the same certification authority (CA) for each:

• Request certificates from the CA.

• Approve the certificate requests on the CA.

• Install the approved certificates in the computer certificate stores.

• Use the MOMCertImport tool to configure Operations Manager 2007.

These are the same steps for installing certificates on a gateway server, except you do not install or run the gateway approval tool. For more information, see the following topics in this security guide:

How to Obtain a Certificate Using Windows Server 2003 Enterprise CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2003 Stand-Alone CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2008 Enterprise CA in Operations Manager 2007

How to Obtain a Certificate Using Windows Server 2008 Stand-Alone CA in Operations Manager 2007

Certificate Generation Wizard

The steps that are necessary to generate, retrieve, and install certificates are in this Security Guide. A certificate generation wizard has been designed to simplify this process. For more information, see the blog post Obtaining Certificates for Non-Domain Joined Agents Made Easy With Certificate Generation Wizard (https://go.microsoft.com/fwlink/?LinkId=128392).

Confirming Certificate Installation

If you have properly installed the certificate, the following event is written into the Operations Manager event log.

During the setup of a certificate, you run the MOMCertImport tool. When the MOMCertImport tool has finished, the serial number of the certificate that you imported is written to the registry at the following subkey.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Machine Settings

Authentication and Data Encryption Between Root Management Server, Management Server, Gateway Server, and Agents

Communication among these Operations Manager components begins with mutual authentication. If certificates are present on both ends of the communications channel, then certificates will be used for mutual authentication; otherwise, the Kerberos version 5 protocol is used. If any two components are separated across an untrusted domain, mutual authentication must be performed using certificates.

Normal communications, such as events, alerts, and deployment of a management pack, occur over this channel. The previous illustration shows an example of an alert being generated on one of the agents that is routed to the root management server (RMS). From the agent to the gateway server, the Kerberos security package is used to encrypt the data, because the gateway server and the agent are in the same domain. The alert is decrypted by the gateway server and re-encrypted using certificates for the management server. After the management server receives the alert, the management server decrypts the message, re-encrypts it using the Kerberos protocol, and sends it to the RMS where the RMS decrypts the alert.

Some communication between the RMS and the agent may include credential information; for example, configuration data and tasks. The data channel between the agent and the management server adds another layer of encryption in addition to the normal channel encryption. No user intervention is required.

Root Management Server and Operations Manager Database

Run As Account information is stored in an encrypted form in the Operations Manager Database using a symmetric key pair that was created by Operations Manager 2007. If the root management server (RMS) were to need replacing, the new RMS would not be able to read any of the encrypted data from the database. The SecureStorageBackup tool, included with Operations Manager 2007, is used to back up and restore this encryption key.

For information about recovering from disasters involving the loss of the root management server with or without the backup of the encryption key, see the Knowledge Base article titled The Root Management Server encryption key is unavailable after you replace or reinstall the Root Management Server server in Microsoft System Center Operations Manager 2007 (https://go.microsoft.com/fwlink/?LinkId=112310).

Root Management Server and Operations Console, Web Console Server, and Reporting Server

Authentication and data encryption between the root management server (RMS) and the Operations console, Web console server, or Reporting Server is accomplished by using Windows Communication Foundation (WCF) technology (formerly code-named "Indigo"). The initial attempt at authentication is made by using the user's credentials. The Kerberos protocol is attempted first. If the Kerberos protocol does not work, another attempt is made using NTLM. If authentication still fails, the user is prompted to provide credentials. After authentication has taken place, the data stream is encrypted as a function of either the Kerberos protocol or SSL, if NTLM is used.

In the case of a Reporting Server and an RMS, after authentication has occurred, a data connection is established between the RMS and SQL Server Reporting Server. This is accomplished by strictly using the Kerberos protocol; therefore, the RMS and Reporting Server must reside in trusted domains. For more information about WCF, see the MSDN article What Is Windows Communication Foundation? (https://go.microsoft.com/fwlink/?LinkId=87429).

Management Server and Reporting Data Warehouse

Two communication channels exist between a management server and the Reporting data warehouse:

• The monitoring host process spawned by the health service (System Center Management service) in either a management server or a root management server

• The SDK service (System Center Data Access services) in the root management server

Monitoring Host Process and Reporting Data Warehouse

By default, the monitoring host process spawned by the Health Service, which is responsible for writing collected events and performance counters to the data warehouse, achieves Windows Integrated Authentication by running as the Data Writer Account specified during Reporting Setup. The account credential is securely stored in a Run As Account called Data Warehouse Action Account. This Run As Account is a member of a Run As Profile called Data Warehouse Account (which is associated with the actual collection rules).

If the Reporting data warehouse and the management server are separated by a trust boundary (for example, each resides in different domains with no trust), then Windows Integrated Authentication will not work. To work around this situation, the monitoring host process can connect to the Reporting data warehouse using SQL Server Authentication. To do this, create a new Run As Account (of Simple Account type) with the SQL account credential and make it a member of the Run As Profile called Data Warehouse SQL Server Authentication Account, with the management server as the target computer.

The following outlines the relationship of the various account credentials, Run As Accounts, and Run As Profiles for both Windows Integrated Authentication and SQL Server Authentication.

Default: Windows Integrated Authentication

Run As Profile: Data Warehouse Account

     Run As Account: Data Warehouse Action Account

          Credentials: Data Writer Account (specified during setup)

Run As Profile: Data Warehouse SQL Server Authentication Account

     Run As Account: Data Warehouse SQL Server Authentication Account

          Credentials: Special account created by Operations Manager (do not change)

Optional: SQL Server Authentication

Run As Profile: Data Warehouse SQL Server Authentication Account

     Run As Account: A Run As Account you create.

          Credentials: An account you create.

The System Center Data Access Service or the SDK Service, and Reporting Data Warehouse

The SDK service found in Operations Manager 2007 SP1 is renamed to the System Center Data Access service in Operations Manager 2007 R2.

By default, the System Center Data Access service, or SDK service, which is responsible for reading data from the Reporting data warehouse and making it available in the Report Parameter Area, achieves Windows Integrated Authentication by running as the SDK and Config account that was defined during setup of Operations Manager 2007.

If the Reporting data warehouse and the management server are separated by a trust boundary (for example, each resides in different domains with no trust), then Windows Integrated Authentication would not work. To work around this situation, the System Center Data Access service or SDK service can connect to the Reporting data warehouse using SQL Server Authentication. To do this, create a new Run As Account (of Simple Account type) with the SQL account credential and make it a member of the Run As Profile called Reporting SDK SQL Server Authentication Account with the management server as the target computer.

The following outlines the relationship of the various account credentials, Run As Accounts, and Run As Profiles for both Windows Integrated Authentication and SQL Server Authentication.

Default: Windows Integrated Authentication

SDK and Config Service Account (defined during setup of Operations Manager)

Run As Profile: Reporting SDK SQL Server Authentication Account

     Run As Account: Reporting SDK SQL Server Authentication Account

          Credentials: Special account created by Operations Manager (do not change)

Optional: SQL Server Authentication

Run As Profile: Data Warehouse SQL Server Authentication Account

     Run As Account: A Run As Account you create.

          Credentials: An account you create.

Operations Console and Reporting Server

The Operations console connects to Reporting Server on port 80 using HTTP. Authentication is performed by using Windows Authentication. Data can be encrypted by using the SSL channel. For more information about using SSL between the Operations console and Reporting Server, see How to Configure the Operations Console to Use SSL When Connecting to a Reporting Server in Operations Manager 2007 later in the Security Guide.

Reporting Server and Reporting Data Warehouse

Authentication between Reporting Server and the Reporting data warehouse is accomplished using Windows Authentication. The account that was specified as the Data Reader Account during setup of Reporting becomes the Execution Account on Reporting Server. If the password for the account should change, you will need to make the same password change using the Reporting Services Configuration Manager in SQL Server 2005. For more information about resetting this password, see How to Change the Reporting Server Execution Account Password in Operations Manager 2007. The data between the Reporting Server and the Reporting data warehouse is not encrypted.

See Also

Tasks

How to Change the Reporting Server Execution Account Password in Operations Manager 2007
How to Configure the Operations Console to Use SSL When Connecting to a Reporting Server in Operations Manager 2007
How to Obtain a Certificate Using Windows Server 2003 Enterprise CA in Operations Manager 2007
How to Obtain a Certificate Using Windows Server 2003 Stand-Alone CA in Operations Manager 2007
How to Obtain a Certificate Using Windows Server 2008 Enterprise CA in Operations Manager 2007
How to Obtain a Certificate Using Windows Server 2008 Stand-Alone CA in Operations Manager 2007
How to Remove Certificates Imported with MOMCertImport in Operations Manager 2007