How to Configure Mutual TLS for Domain Security

How to Configure Mutual TLS for Dom...

Applies to: Exchange Server 2007 SP1, Exchange Server 2007 Topic Last Modified: 2009-04-20

This topic explains how to use the Exchange Management Shell to configure mutual Transport Layer Security (TLS) for Domain Security, the set of functionality in Microsoft Exchange Server 2007 and Microsoft Office Outlook 2007 that provides a relatively low cost alternative to S/MIME and other message-level security solutions.

For the purposes of illustration, this topic explains how Exchange administrators at a fictitious company, Contoso, configure their Exchange 2007 environment to exchange domain-secured e-mail with their partner, Woodgrove Bank. In this scenario, Contoso wants to make sure that all e-mail that is sent and received from Woodgrove Bank is protected with mutual TLS. Also, Contoso wants to configure Domain Security functionality so that all mail to and from Woodgrove Bank is rejected if mutual TLS cannot be used.

Contoso has an internal public key infrastructure (PKI) that generates certificates. The PKI's root certificate has been signed by a major third-party certification authority (CA). Woodgrove Bank uses the same third-party CA to generate their certificates. Therefore, both Contoso and Woodgrove Bank each trust the other's root CAs.

To set up mutual TLS, Exchange administrators at Contoso perform the following procedures:

Generate a certificate request for TLS certificates.
Import the certificate to Edge Transport servers.
Configure outbound domain security.
Configure inbound domain security.
Test mail flow.

Before You Begin

The configuration of mutual TLS requires the following:

Access to internal Exchange 2007 servers by using the Set-TransportConfig cmdlet and by using the New-SendConnector cmdlet if you haven't configured Send connectors.
Access to the Edge Transport Server computers where the ExchangeCertificate cmdlets are run.

Generally, configuration changes that are made to Domain Security functionality that don't use the ExchangeCertificate cmdlets should be made within the organization and synchronized to Edge Transport servers by using the Microsoft Exchange EdgeSync service.

When you import and configure TLS certificates by using the ExchangeCertificate cmdlets, you must run the cmdlets on the Edge Transport server that you are configuring. To run the ExchangeCertificate cmdlets on a computer that has the Edge Transport server role installed, you must log on by using an account that is a member of the local Administrators group on that computer.

To run the Set-TransportConfig cmdlet, the account you use must be delegated the Exchange Organization Administrator role.

To run the New-SendConnector cmdlet, the account you use must be delegated the Exchange Server Administrator role and local Administrators group on that computer.

For more information about permissions, delegating roles, and the rights that are required to administer Exchange 2007, see Permission Considerations.

This topic assumes that you have read and understood Creating a Certificate or Certificate Request for TLS.

The Microsoft Exchange EdgeSync service must be fully deployed for Domain Security.

Before you can successfully run mutual TLS on an Edge Transport server, you must configure the computer and PKI environment so that certificate validation and certificate revocation list checking are operable. For more information, see How to Enable PKI on the Edge Transport Server for Domain Security.

Generating a Certificate Request for TLS Certificates

As mentioned earlier in this topic, Contoso has an internal PKI that is subordinated to a third-party CA. In this scenario, subordination refers to the fact that the CA that is deployed by Contoso in their corporate infrastructure contains a root certificate that has been signed by a public third-party CA. By default, the public third-party CA is one of the trusted root certificates in the Microsoft Windows certificate store. Therefore, any client that includes the same third-party CA in its trusted root store and that connects to Contoso can authenticate to the certificate that is presented by Contoso.

Contoso has two Edge Transport servers that require TLS certificates: mail1.contoso.com and mail2.mail.contoso.com. Therefore, the Contoso e-mail administrator must generate two certificate requests, one certificate request for each server.

The following procedure shows the command that the administrator uses to generate a base64-encoded PKCS#10 certificate request. The Contoso administrator must run this command two times: one time for CN=mail1.contoso.com and one time for CN=mail2.mail.contoso.com.

Note:
The common name (CN) in the Subject name of the resulting certificates is mail1.contoso.com and mail2.mail.contoso.com respectively. The Subject Alternative Name contains "mail.contoso.com," which is the fully qualified domain name (FQDN) of one of the accepted domains that is configured for Contoso.

To create a TLS certificate request

Run the following command:

New-ExchangeCertificate -GenerateRequest -FriendlyName "Internet certificate" -Path c:\certificates\request.p7c -SubjectName "DC=com,DC=Contoso,CN=mail1.contoso.com"  -DomainName mail.contoso.com

For detailed syntax and parameter information, see New-ExchangeCertificate.

Important:
The specific details of the certificate or certificate request that you create depends on many variables. If you are generating a request, make sure that you work closely with the CA or the PKI administrator who will issue your certificate. For more information about how to create a certificate request for TLS, see Creating a Certificate or Certificate Request for TLS.

Transporting Certificates and Related Keys

When you receive a certificate from your PKI or CA provider, convert the issued certificate to a PFX (PKCS#12) file so that you can back it up as part of a disaster contingency. A PFX file contains the certificate and related keys. In some cases, you may want to transport the certificate and keys to move them to other computers. For example, if you have multiple Edge Transport servers where you expect to send and receive e-mail that is Domain Secured, you can create a single certificate that will work for all servers. In this case, you must have the certificate imported and enabled for TLS on each Edge Transport server.

As long as you keep a copy of the PFX file securely archived, you can always import and enable the certificate. The PFX file contains the private key so it's important to physically protect the file by keeping in on storage media in a secure location.

It's important to understand that the Import-ExchangeCertificate cmdlet always marks the imported private key from the PFX as non-exportable. This functionality is by design.

When you use the Certificate Manager snap-in in Microsoft Management Console (MMC) to import a PFX file, you can specify private key exportability and strong-key protection.

Important:
Do not enable strong-key protection on certificates that are intended for TLS. Strong-key protection prompts the user every time that the private key is accessed. With Domain Security, the "user" is the SMTP service on the Edge Transport server.

Importing Certificates to Edge Transport Servers

After the administrator generates the certificate requests, the administrator then uses the requests to generate the certificates for the servers. The resulting certificates must be issued as either a single certificate or a certificate chain and copied to the appropriate Edge Transport servers.

Important:
To import the certificates, you must use the Import-ExchangeCertificate cmdlet. For more information, see Import-ExchangeCertificate.

Important:
Do not use the Certificate Manager snap-in in MMC to import the certificates for TLS on the Exchange server. Using the Certificate Manager snap-in to import certificates on Exchange servers does not bind the request that is created in this procedure to the issued certificate. Therefore TLS might fail. You can use the Certificate Manager snap-in to import certificates and keys that are stored as PFX files into the local computer store. After the certificates are imported, you can then enable the certificate for TLS by running the Enable-ExchangeCertificate cmdlet.

Important:

Do not use the Certificate Manager snap-in in MMC to import the certificates for TLS on the Exchange server. Using the Certificate Manager snap-in to import certificates on Exchange servers does not bind the request that is created in this procedure to the issued certificate. Therefore TLS might fail. You can use the Certificate Manager snap-in to import certificates and keys that are stored as PFX files into the local computer store. After the certificates are imported, you can then enable the certificate for TLS by running the Enable-ExchangeCertificate cmdlet.

When you import the certificate to the Edge Transport server, you must also enable the certificate for the Simple Mail Transfer Protocol (SMTP) service. The Contoso administrator runs the following command on each Edge Transport server, one time for each respective certificate.

To import and enable a TLS certificate for Domain Security on an Edge Transport server

Run the following command:

Import-ExchangeCertificate -Path c:\certificates\import.pfx | Enable-ExchangeCertificate -Services SMTP

This command imports and enables the TLS certificate by pipelining the Enable-ExchangeCertificate cmdlet.

You can also enable the certificate after you import it. To import the certificate, run the following command:

Import-ExchangeCertificate -Path c:\certificates\import.pfx

Then, copy the thumbprint to the Windows clipboard. To display the thumbprint of the certificate that you have just imported, run the following command:

Get-ExchangeCertificate |FL

Copy the data from the Thumbprint field to the Windows clipboard.

Finally, to enable the certificate, run the following command:

Enable-ExchangeCertificate -Thumbprint AB493A0C9A6C0327162FE04EF74609CB8FB7FE24 -Services SMTP

Configuring Outbound Domain Security

You must perform three steps to configure outbound Domain Security:

Run the Set-TransportConfig cmdlet to specify the domain with which you want to send domain-secured e-mail.
Run the Set-SendConnector cmdlet to set the DomainSecureEnabled property on the Send connector that will send mail to the domain with which you want to send domain-secured e-mail.
Run the Set-SendConnector cmdlet to verify the following:
- The Send connector that will send mail to the domain with which you want to send domain-secured e-mail routes mail with Domain Name System (DNS).
- The FQDN is set to match either the Subject Name or the Subject Alternative Name of certificates that you are using for Domain Security.

Because the changes that you make in these three steps are global, you must make the changes on an internal Exchange server. The configuration changes that you make will be replicated out to the Edge Transport servers by using the Microsoft Exchange EdgeSync service.

Specifying the Sender Domain in Transport Configuration

It is relatively straightforward to specify the domain with which you want to send domain-secured e-mail. The Contoso administrator runs the following command on an internal Exchange 2007 server:

Set-TransportConfig -TLSSendDomainSecureList woodgrovebank.com

For more information, see Set-TransportConfig.

Note:
The TLSSendDomainSecureList parameter takes a multivalued list of domain names. The Set-TransportConfig command replaces the whole value for TLSSendDomainSecureList with the new value that is supplied in the cmdlet. Therefore, if you want to add a new domain, the value that you supply must include the existing list and the new domain.

Configuring a Send Connector

Contoso will use their default DNS-routed Send connector to send domain-secured e-mail to their partners. Because their default DNS-routed Send connector is a default Internet Send connector, it uses DNS to route mail and does not use a smart host. The FQDN is already set to mail.contoso.com. Because the certificates that the Contoso administrator created set the DomainName parameter of the New-ExchangeCertificate to mail.contoso.com, the Send connector can use the certificates without additional configuration.

If you have configured a subdomain for testing, you may have to update the FQDN of your Send connector to match the certificate that you have created (for example, subdomain.mail.contoso.com). On the other hand, if you have created a certificate that contains the subdomain in the Subject or Subject Alternative name Fields, you don't have to update the FQDN of the Send connector.

Therefore, the only configuration that the Contoso administrator must make to the Send connector is to set the DomainSecureEnabled parameter. To do this, the Contoso administrator runs the following command on an internal Exchange 2007 server for the "Internet" Send connector:

Set-SendConnector Internet -DomainSecureEnabled:$True

For more information, see Set-SendConnector.

You can also use the Exchange Management Console to enable domain-secured e-mail on a Send connector.

To use the Exchange Management Console to configure a Send connector for Domain Security

On a Hub Transport server, open the Exchange Management Console, click Organization Configuration, click Hub Transport, and then in the result pane, click the Send Connectors tab.
Select the Send connector that sends mail to the domain from which you want to send domain-secured e-mail, and then, in the action pane, click Properties.
On the Network tab, select Enable Domain Security (Mutual Auth TLS), click Apply, and then click OK.

Configuring Inbound Domain Security

You must perform two steps to enable inbound Domain Security:

Run the Set-TransportConfig cmdlet to specify the domain from which you want to receive domain-secured e-mail.
On the Edge Transport server, use the Exchange Management Shell or the Exchange Management Console to enable Domain Security on the Receive connector from which you want to receive domain-secured e-mail. Because Domain Security requires mutual TLS authentication, TLS must also be enabled on the Receive connector.

Specifying the Recipient Domain in Transport Configuration

It is relatively straightforward to specify the domain with which you want to receive domain-secured e-mail. To specify the domain, the Contoso administrator runs the following command on an internal Exchange 2007 server:

Set-TransportConfig -TLSReceiveDomainSecureList woodgrovebank.com

For more information, see Set-TransportConfig.

Note:
The TLSReceiveDomainSecureList parameter takes a multivalued list of domain names. The Set-TransportConfig command replaces the whole value for TLSReceiveDomainSecureList parameter with the new value supplied by the Set-TransportConfig cmdlet. Therefore, if you want to add a new domain, the value that you supply must include the existing list and the new domain.

Configuring a Receive Connector

You must configure the Receive connector on each Edge Transport server that accepts mail from the domain from which you want to receive domain-secured e-mail. The Contoso environment is configured to have a single Internet Receive connector, with an Identity of Inet, on both Edge Transport servers. Therefore, to enable TLS while mail is sent to or received from Woodgrove Bank, the Contoso administrator must make sure that TLS is enabled on the default Internet Receive connector on both Edge Transport servers.

To use the Exchange Management Shell to configure a Receive connector for Domain Security

On the Edge Transport server, run the following command:

Set-ReceiveConnector Inet -DomainSecureEnabled:$True -AuthMechanism TLS

For detailed syntax and parameter information, see Set-ReceiveConnector.

To use the Exchange Management Console to configure a Receive connector for Domain Security

On the Edge Transport server, open the Exchange Management Console, click Edge Transport, and then in the result pane, click the Receive Connectors tab.
Select the Receive connector that accepts mail from the domain from which you want to receive domain-secured e-mail, and then, in the action pane, click Properties.
On the Authentication tab, select Transport Layer Security (TLS) and Enable Domain Security (Mutual Auth TLS), and then click OK.

Be aware that specifying the authentication mechanism as TLS doesn't force TLS on all inbound connections.

TLS will be forced for connections from Woodgrove Bank for the following reasons:

Woodgrove Bank is specified in the Set-TransportConfig cmdlet on the TLSReceiveDomainSecureList parameter.
The DomainSecureEnabled parameter is set to $True on the Receive connector.

Other senders that are not listed on the TLSReceiveDomainSecureList parameter in the Set-TransportConfig cmdlet will only use TLS if TLS is supported by the sending system.

Note:
A side effect of enabling Domain Security on a Receive connector is that a client certificate is requested during TLS negotiation. If the domain from which the client is sending is not in the secure domain list, the client will not be required to submit the certificate.

Testing Domain-Secured Mail Flow

After you have configured domain-secured e-mail, you can test the connection by reviewing the performance logs and the protocol logs. Messages that have successfully authenticated over the domain-secured mail flow path are displayed in Outlook 2007 as "Domain Secure" messages.

Performance Counters

The Domain Security feature includes the following set of performance counters under MSExchange Secure Mail Transport:

Domain-Secured Messages Received
Domain-Secured Messages Sent
Domain-Secured Outbound Session Failures

You can create a new counter log file for domain-secured mail flow with these performance counters to monitor the number of messages sent and received and also to monitor failed mutual TLS sessions. For more information about how to create and configuring counter logs, see the Help file that is included with the Performance Logs and Alerts MMC snap-in.

Protocol Logs

You can review the send and receive protocol logs to determine whether TLS negotiation has been successful. Successful TLS negotiation will generate logs that resemble the following examples.

To view detailed protocol logs, you must set the protocol logging level to Verbose on the connectors that your organization uses to send and receive Domain Secured e-mail.

To use the Exchange Management Shell to configure a Receive connector for verbose protocol logging

On the Edge Transport server, run the following command:
```
Set-ReceiveConnector Inet -ProtocolLoggingLevel Verbose
```
Where Inet is the name of the Receive connector on which Domain Secured e-mail is enabled.

To use the Exchange Management Shell to configure a Send connector for verbose protocol logging

On the Edge Transport server, run the following command:
```
Set-SendConnector Internet -ProtocolLoggingLevel Verbose
```
Where Internet is the name of the Send connector where Domain Secured e-mail is enabled.

Example of a Send Log

<220 edgedns3 ESMTP Microsoft ESMTP MAIL Service, Version: 8.0.647.0; Tue, 29 Aug 2006 04:22:00 -0700 (PDT)
>EHLO edgea36.dns.contoso.com
<250-edgedns3 Hello woodgrove.com [192.168.0.2], pleased to meet you
<250-ENHANCEDSTATUSCODES
<250-PIPELINING
<250-EXPN
<250-VERB
<250-8BITMIME
<250-SIZE
<250-DSN
<250-ETRN
<250-STARTTLS
<250-DELIVERBY
<250 HELP
>STARTTLS
<220 2.0.0 Ready to start TLS
*Sending certificate
*CN=edgea36, Certificate subject
*CN=edgea36, Certificate issuer name
*CA2EDF2487C6F09B4E413FD3812A7F89, Certificate serial number
*E8DA062786FD097DD8D79FF10C583CC23AD64F6C, Certificate thumbprint
*edgea36;edgea36.dns.contoso.com, Certificate alternate names
*Received certificate
*CN=smi.extest.contoso.com, OU=Contoso, O=Corp, L=Spokane, S=WA, C=US, Certificate subject
*CN=ExCertDom EntSub Issuing CA v1.0, DC=ExCertDom, DC=ExTest, DC=Contoso, DC=Com, Certificate issuer name
*446DD186000A00002819, Certificate serial number
*DC27B5F8657F84B15B5004BE63CE482721871582, Certificate thumbprint
*smi.extest.contoso.com, Certificate alternate names
>EHLO edgea36.dns.contoso.com
<250-edgedns3 Hello woodgrove.com [192.168.0.2], pleased to meet you
<250-ENHANCEDSTATUSCODES
<250-PIPELINING
<250-EXPN
<250-VERB
<250-8BITMIME
<250-SIZE
<250-DSN
<250-ETRN
<250-DELIVERBY
<250 HELP
*08C895F533E837EC;2006-08-28T22:37:53.323Z;1, sending message
>MAIL FROM:<user@example.com> SIZE=614
>RCPT TO:<root@smi.extest.contoso.com>
<250 2.1.0 <user@example.com>... Sender ok
<250 2.1.5 <root@smi.extest.contoso.com>... Recipient ok
>DATA
<354 Enter mail, end with "." on a line by itself
<250 2.0.0 k7TBM0BZ000043 Message accepted for delivery
>QUIT
<221 2.0.0 edgedns3 closing connection

Example of a Receive Log

>220 edgea36 Microsoft ESMTP MAIL Service, Version: 8.0.647.0 ready at Mon, 28 Aug 2006 15:37:53 -0700
<EHLO edgedns3
>250-edgea36.dns.contoso.com Hello [192.168.0.1]
>250-SIZE 15728640
>250-PIPELINING
>250-DSN
>250-ENHANCEDSTATUSCODES
>250-STARTTLS
>250-AUTH
>250-8BITMIME
>250-BINARYMIME
>250 CHUNKING
<STARTTLS
>220 2.0.0 SMTP server ready
*Sending certificate
*CN=edgea36, Certificate subject
*CN=edgea36, Certificate issuer name
*CA2EDF2487C6F09B4E413FD3812A7F89, Certificate serial number
*E8DA062786FD097DD8D79FF10C583CC23AD64F6C, Certificate thumbprint
*edgea36;edgea36.dns.contoso.com, Certificate alternate names
*Received certificate
*CN=smi.extest.contoso.com, OU=Contoso, O=Corp, L=Spokane, S=WA, C=US, Certificate subject
*CN=ExCertDom EntSub Issuing CA v1.0, DC=ExCertDom, DC=ExTest, DC=Contoso, DC=Com, Certificate issuer name
*446DD186000A00002819, Certificate serial number
*DC27B5F8657F84B15B5004BE63CE482721871582, Certificate thumbprint
*smi.extest.contoso.com, Certificate alternate names
<EHLO edgedns3
>250-edgea36.dns.contoso.com Hello [192.168.0.1]
>250-SIZE 15728640
>250-PIPELINING
>250-DSN
>250-ENHANCEDSTATUSCODES
>250-AUTH
>250-8BITMIME
>250-BINARYMIME
>250 CHUNKING
<MAIL From:<user@smi.extest.contoso.com> SIZE=16
*08C895F533E837EC;2006-08-28T22:37:53.323Z;1, receiving message
>250 2.1.0 user@smi.extest.contoso.com Sender OK
<RCPT To:<user@woodgrove.com>
>250 2.1.5 user@woodgrove.com Recipient OK
<DATA
>354 Start mail input; end with <CRLF>.<CRLF>
>250 2.6.0 <200608281121.k7SBHYi0004586@edgedns3> Queued mail for delivery
<QUIT
>221 2.0.0 Service closing transmission channel

For more information about how to view protocol logs, see How to Configure Protocol Logging.

For more information about how to manage Domain Security in Exchange 2007, see Managing Domain Security.

For more information about how to configure a public key infrastructure, see Best Practices for Implementing a Microsoft Windows Server 2003 Public Key Infrastructure.