How to Use TLS Authentication in Exchange 2007 to Send and Receive Messages with Third-Party E-Mail

  • Article

Microsoft Exchange Server 2007 will reach end of support on April 11, 2017. To stay supported, you will need to upgrade. For more information, see Resources to help you upgrade your Office 2007 servers and clients.

 

Applies to: Exchange Server 2007 SP3

This topic describes how to use Transport Layer Security (TLS) authentication together with Microsoft Exchange Server 2007 to send and receive e-mail messages with a third-party e-mail program.

By using the TLS protocol, you can help improve the security of SMTP communication in Exchange 2007. TLS is a standard protocol that is used to provide secure Web communications on the Internet or on intranets. TLS enables clients to authenticate servers or, optionally, servers to authenticate clients. It also provides a security channel by encrypting communications. TLS is the latest version of the Secure Sockets Layer (SSL) protocol.

TLS over SMTP offers certificate-based authentication and helps provide security-enhanced data transfers by using symmetric encryption keys. In symmetric-key encryption, also known as "shared secret" encryption, the same key is used to encrypt and to decrypt the message. TLS applies a hash-based Message Authentication Code (HMAC). HMAC uses a hash algorithm in combination with a shared secret key to help make sure that the data has not been changed during transmission. The shared secret key is appended to the data to be hashed. This helps improve the security of the hash because both parties must have the same shared secret key to verify that the data is authentic.

In earlier versions of Exchange, you had to configure TLS manually. Additionally, you had to install a valid certificate, suitable for TLS usage, on the Exchange server. In Exchange 2007, Setup creates a self-signed certificate. By default, TLS is enabled. This enables any sending system to encrypt the incoming SMTP session to Exchange. By default, Exchange 2007 also tries TLS for all remote connections.

To use TLS to send e-mail messages to a third-party e-mail program, you must configure a Send connector. Send connectors are configured on computers that are running Exchange 2007 and that have Hub Transport or Edge Transport server roles installed. The Send connector represents a logical gateway through which outgoing messages are sent.

To use TLS to send e-mail messages to a third-party e-mail program, you must configure a Receive connector. Receive connectors are configured on computers that are running Exchange 2007 and that have the Hub Transport or Edge Transport server roles installed. Receive connectors represent a logical gateway through which all inbound messages are received.

To use TLS to send e-mail messages to a third-party e-mail program

  1. Start Exchange Management Console.

  2. Perform one of the following steps:

    • On a computer that has the Edge Transport server role installed, select Edge Transport, and then click the Send Connectors tab.

    • To create a Send connector on a Hub Transport server role, in the console tree, expand Organization Configuration, select Hub Transport, and then click the Send Connectors tab.

  3. In the action pane, click New Send Connector. The New SMTP Send Connector Wizard starts.

  4. On the Introduction page type a meaningful name for the connector in the Name field. This name is used to identify the connector.

  5. In the Select the intended use for this connector list, click Custom, and then click Next. 

  6. On the Address space page, click Add.

  7. In the SMTP Address Space dialog box, type the external domain of the third-party e-mail server. For example, type *.contoso.com for the contoso.com domain.

  8. Click OK, and then click Next.

  9. On the Network settings page, click Use domain name system (DNS) "MX" records to route mail automatically, and then click Next. Or, click Route all mail through the following smart hosts, and then follow these steps:

    1. Click Add.

    2. In the Add Smart Host dialog box, select IP Address or Fully qualified domain name (FQDN) to specify how to locate the smart host. If you select IP Address, enter the IP address of the smart host. If you select Fully qualified domain name (FQDN), enter the FQDN of the smart host. The sending server must be able to resolve the FQDN.

    3. When you are finished, click OK.

    4. To add more smart hosts, click Add, and repeat steps b and c.

    5. To edit the settings of a smart host, select the smart host, and then click Edit.

    6. To remove an existing smart host, select the smart host, and then click Remove icon.

    7. When you are finished, click Next.

    8. On the Smart host security settings page, select Basic Authentication over TLS, and then click Next.

  10. By default, the Hub Transport server that you are currently working on is listed as a source server on the Source Server page. To add a source server, click Add. In the Select Hub Transport servers and Edge Subscriptions dialog box, select the Hub Transport servers or the subscribed Edge Transport servers that will be used as the source servers for sending messages to the address space that you provided in step 7. The list of source servers can contain all Hub Transport servers or all subscribed Edge Transport servers, but not a mix of both. When you are finished adding additional source servers, click OK.

    To add more source servers, click Add, and repeat this step.

    To remove an existing source server, select the source server, and then click Remove icon.

    When you are finished, click Next.

  11. On the New connector page, review the configuration summary for the connector. If you want to change the settings, click Back. To create the Send connector by using the settings in the configuration summary, click New.

  12. On the Completion page, click Finish.

  13. Some third-party programs, such as Gentoo Linux, do not require more configurations. Test the connection. If a connection cannot be completed, follow these steps:

    1. In the work pane, right-click the connector you created, and then click Properties.

    2. On the Network tab, click to select the Enable Domain Security (Mutual Auth TLS) check box, and then click OK.

    3. Close the Exchange Management Console.

    4. Restart the Microsoft Exchange Transport service.

To use TLS to receive e-mail messages from a third-party e-mail program

  1. Start Exchange Management Console.

  2. Perform one of the following steps:

    1. On a computer that has the Edge Transport server role installed, select Edge Transport, and then in the work pane, click the Receive Connectors tab.

    2. To create a Receive connector on a Hub Transport server role, in the console tree, expand Server Configuration, and then click Hub Transport. In the results pane, select the server on which you want to create the connector, and then click the Receive Connectors tab.

  3. In the action pane, click New Receive Connector. The New SMTP Receive Connector Wizard starts.

  4. On the Introduction page, type a meaningful name for the connector in the Name field. This name is used to identify the connector.

  5. In the Select the intended use for this connector list, click Custom, and then click Next.

  6. On the Local network settings page, click Add.

  7. In the Add Receive Connector Binding dialog box, select one of the following options:

    • Use all IP addresses available on this server   If you select this option, the connector listens for connections on all the IP addresses that are assigned to the network adapters on the local server.

    • Specify an IP address   If you select this option, you must type an IP address that is assigned to a network adapter on the local server. The connector listens for connections only on the IP address that you provide.

      Note

      You must specify a local IP address that is valid for the Hub Transport server or Edge Transport server on which the Receive connector is located. If you specify an invalid local IP address, the Microsoft Exchange Transport service may not start when the service is restarted.

  8. On the Local network settings page, in the Port field, type a port number, and then click OK.
    To add multiple local IP addresses to this connector, click Add and repeat this step.
    To change a previous entry, select the entry, and then click Edit. To remove an existing entry, select the entry, and then click Remove icon.

  9. On the Local network settings page, in the Specify the FQDN this connector will provide in response to HELO or EHLO field, type the name that is advertised in response to the SMTP HELO or EHLO verb. If you leave this field blank, the fully qualified domain name (FQDN) of the Hub Transport server or Edge Transport server is automatically added when the connector is created. Click Next.

  10. On the Remote network settings page, type the IP address or IP address range of the third-party program from which the connector will accept incoming connections. To add the remote IP address or remote IP address range, use one of the following methods:

    • To enter an IP address or subnet without a subnet mask, or to specify the subnet mask by using Classless Interdomain Routing (CIDR) notation, click Add or the drop-down arrow located next to Add and select IP Address. In the Add IP address(es) of Remote Servers dialog box, enter the IP address by using one of the following methods:

      IP address without a subnet mask   For example, type 192.168.1.0. If you do not specify a subnet mask by using CIDR notation, the classful default subnet mask is assumed.

      IP address by using CIDR notation   For example, type 192.168.1.0/24.

    • To enter an IP address or subnet together with a subnet mask in dotted decimal notation, click the drop-down arrow located next to Add, and then click IP and Mask. In the Add Remote Servers - IP and Mask dialog box, type the IP address and the subnet mask by using the following syntax:

      IP Address   For example, type 192.168.1.0.

      Subnet Mask   For example, type 255.255.255.0.

    • To specify an IP address range by using the first IP address and the last IP address in the range, click the drop-down arrow located next to Add, and then click IP Range. In the Add Remote Servers - IP Range dialog box, type the IP address by using the following syntax:

      Start Address   For example, type 192.168.1.1.

      End Address   For example, type 192.168.255.255.

      Because you cannot specify a subnet mask, the classful default subnet mask is assumed.

    When you are finished, click OK. To add multiple remote network ranges to this connector, click Add and repeat this step. To change a previous entry, select the entry, and then click Edit. To remove an existing entry, select the entry, and then click Remove icon.

  11. When you are finished, click Next.

  12. On the New Connector page, review the configuration summary for the connector. If you want to change the settings, click Back. To create the Receive connector by using the settings in the configuration summary, click New.

  13. On the Completion page, click Finish.

  14. In the work pane, right-click the connector you created, and then click Properties.

  15. On the Authentication tab, click to select the Enable Domain Security (Mutual Auth TLS) check box if one of the following conditions is true:

    • The sending server and the receiving server are both using a public certificate from a trusted certificate issuer.

    • The sending server and the receiving server are both using a self-issued certificate with each other's root certificate installed as the trusted root certificate.

  16. On the Permission Groups tab, click to select the Anonymous users check box, and then click OK.

  17. Close the Exchange Management Console.

  18. Start Exchange Management Shell.

  19. Run the following cmdlet:

    Set-ReceiveConnector  -identity  <ReceiveConnectorIdParameter> -RequireTLS $true -AuthenticationMechanism TLS

  20. If one of the following conditions is true:

    • The sending server and the receiving server are both using a public certificate from a trusted certificate issuer.

    • The sending server and the receiving server are both using a self-issued certificate with each other's root certificate installed as the trusted root certificate.

    Run the following cmdlet:

    Set-TransportConfig -TLSReceiveDomainSecureList <remotedomain>.com, <remotedomain>.net

  21. Restart the Microsoft Exchange Transport service.

More Information

For more information about Send connectors, see the Send Connectors and How to Create a New Send Connector topics.

For more information about Receive connectors, see the Receive Connectors and How to Create a New Receive Connector topics.

For more information about the Set-ReceiveConnector cmdlet, see the Set-ReceiveConnector topic.

For more information about the Set-TransportConfig cmdlet, see the Set-TransportConfig topic.

For more information about how to use the Transport Layer Security (TLS) protocol together with Exchange 2007, see the following topics: