Appendix C: VPN Tunneling Protocols

  • Article

Applies To: Windows Server 2008, Windows Server 2008 R2

Virtual private network (VPN) tunneling protocols enable the encapsulation of a packet from one type of protocol within the datagram of a different protocol. For example, Point-to-Point Tunneling Protocol (PPTP) can encapsulate IP packets over a public network, such as the Internet. A VPN solution can be based on Internet Key Exchange version 2 (IKEv2), PPTP, Layer Two Tunneling Protocol (L2TP), or Secure Socket Tunneling Protocol (SSTP).

IKEv2, PPTP, L2TP, and SSTP depend heavily on the features originally specified for Point-to-Point Protocol (PPP). PPP was originally defined as the protocol to use between a dial-up client and a network access server.

IKEv2

IKEv2 allows multiprotocol traffic to be encrypted and then encapsulated in an IPsec header to be sent across a private IP network or a public IP network, such as the Internet. IKEv2 can be used for remote access VPN connections. IKEv2 supports automatic VPN reconnect that allows the security association to survive changes in the underlying connection. This enables a VPN connection to continue working with no user interaction when the network connection changes from wired to wireless, or if the IP address changes. When using the Internet as the public network for VPN, the IKEv2 server is an IKEv2-enabled VPN server with one interface on the Internet and a second interface on the intranet.

Both IKEv2 and IPsec must be supported by both the VPN client and the VPN server. IKEv2 is supported by remote access clients running Windows 7, and by VPN servers running Windows Server 2008 R2.

Encapsulation

IKEv2 uses IPsec and the Encapsulating Security Protocol (ESP) to encapsulate IP datagrams for tunneled data. IKEv2 uses IPsec encryption to protect the encapsulated data. The following figure shows the structure of an IKEv2 packet containing an IP datagram.

Structure of an IKEv2 Packet Containing an IP Datagram

Encryption

The IKEv2 message is encrypted with one of the supported IPsec encryption algorithms by using encryption keys generated from the IKEv2 negotiation process.

PPTP

PPTP allows multiprotocol traffic to be encrypted and then encapsulated in an IP header to be sent across a private IP network or a public IP network, such as the Internet. PPTP can be used for remote access and site-to-site VPN connections. When using the Internet as the public network for VPN, the PPTP server is a PPTP-enabled VPN server with one interface on the Internet and a second interface on the intranet.

PPTP is supported by remote access clients running Windows XP and later, and by VPN servers running Windows Server 2003 and later.

Encapsulation

PPTP encapsulates PPP frames in IP datagrams for transmission over the network. PPTP uses a TCP connection for tunnel management and a modified version of Generic Routing Encapsulation (GRE) to encapsulate PPP frames for tunneled data. The payloads of the encapsulated PPP frames can be encrypted, compressed, or both. The following figure shows the structure of a PPTP packet containing an IP datagram.

Structure of a PPTP Packet Containing an IP Datagram

Encryption

The PPP frame is encrypted with Microsoft Point-to-Point Encryption (MPPE) by using encryption keys generated from the MS-CHAP v2 or EAP-TLS authentication process. Virtual private networking clients must use the Microsoft Challenge Handshake Authentication Protocol version 2 (MS-CHAP v2) or Extensible Authentication Protocol-Transport Layer Security (EAP-TLS) authentication protocol in order for the payloads of PPP frames to be encrypted. PPTP is taking advantage of the underlying PPP encryption and encapsulating a previously encrypted PPP frame.

L2TP

L2TP allows multiprotocol traffic to be encrypted and then sent over any medium that supports point-to-point datagram delivery, such as IP or asynchronous transfer mode (ATM). L2TP is a combination of PPTP and Layer 2 Forwarding (L2F), a technology developed by Cisco Systems, Inc. L2TP represents the best features of PPTP and L2F.

Unlike PPTP, the Microsoft implementation of L2TP does not use MPPE to encrypt PPP datagrams. Installed with the TCP/IP protocol, L2TP relies on Internet Protocol security (IPsec) in Transport Mode for encryption services. The combination of L2TP and IPsec is known as L2TP/IPsec.

Both L2TP and IPsec must be supported by both the VPN client and the VPN server. Like PPTP, L2TP is supported by remote access clients running Windows XP and later, and by VPN servers running Windows Server 2003 and later.

Encapsulation

Encapsulation for L2TP/IPsec packets consists of two layers.

First layer: L2TP encapsulation

A PPP frame (an IP datagram) is wrapped with an L2TP header and a UDP header.

The following figure shows the structure of an L2TP packet containing an IP datagram.

Structure of an L2TP Packet Containing an IP Datagram

Second layer: IPsec encapsulation

The resulting L2TP message is then wrapped with an IPsec Encapsulating Security Payload (ESP) header and trailer, an IPsec Authentication trailer that provides message integrity and authentication, and a final IP header. The IP header contains the source and destination IP addresses that correspond to the VPN client and VPN server.

The following illustration shows L2TP and IPsec encapsulation for a PPP datagram.

Encryption of L2TP Traffic with IPsec ESP

Encryption

The L2TP message is encrypted with one of the supported IPsec encryption algorithms by using encryption keys generated from the Internet Key Exchange (IKE) negotiation process.

SSTP

Secure Socket Tunneling Protocol (SSTP) is a tunneling protocol that uses the HyperText Transport Protocol Secure (HTTPS) protocol over TCP port 443 to pass traffic through firewalls and Web proxies that might block PPTP, IKEv2, and L2TP/IPsec traffic. SSTP provides a mechanism to encapsulate PPP traffic over the Secure Sockets Layer (SSL) channel of the HTTPS protocol. The use of PPP allows support for strong authentication methods, such as EAP-TLS. SSL provides transport-level security with enhanced key negotiation, encryption, and integrity checking.

When a client tries to establish a SSTP-based VPN connection, SSTP first establishes a bidirectional HTTPS layer with the SSTP server. Over this HTTPS layer, the protocol packets flow as the data payload.

Encapsulation

SSTP encapsulates PPP frames in IP datagrams for transmission over the network. SSTP uses a TCP connection (over port 443) for tunnel management as well as PPP data frames.

Encryption

The SSTP message is encrypted with the SSL channel of the HTTPS protocol.

Choosing between tunneling protocols

When choosing between IKEv2, PPTP, L2TP/IPsec, and SSTP remote access VPN solutions, consider the following:

  • IKEv2 can be used with client computers running Windows 7 and Windows Server 2008 R2 only. By using IPsec, IKEv2-based VPN connections provide strong data confidentiality (encryption), data integrity (proof that the data was not modified in transit), and data origin authentication (proof that the data was sent by the authorized user). A key feature of an IKEv2 VPN connection is VPN Reconnect that enables a VPN connection to tolerate short interruptions in the underlying network layer. For example, if the IP address changes, or if the wired connection is dropped in favor of a wireless connection, the VPN connection quietly and automatically reestablishes itself in the background without requiring any interaction from the user.

  • PPTP can be used with client computers running Windows XP, Windows Server 2003, and later versions of Windows. Unlike L2TP/IPsec, PPTP does not require the use of a public key infrastructure (PKI). By using encryption, PPTP-based VPN connections provide data confidentiality. PPTP-based VPN connections, however, do not provide data integrity (proof that the data was not modified in transit) or data origin authentication (proof that the data was sent by the authorized user).

  • L2TP can be used with client computers running Windows XP, Windows Server 2003, and later versions of Windows. L2TP supports either computer certificates or a preshared key as the authentication method for IPsec. Computer certificate authentication, the recommended authentication method, requires a PKI to issue computer certificates to the VPN server computer and all VPN client computers. By using IPsec, L2TP/IPsec VPN connections provide data confidentiality, data integrity, and data authentication.

    L2TP/IPsec enables computer authentication at the IPsec layer and user-level authentication at the PPP layer.

  • SSTP can be used only with client computers running Windows Vista with Service Pack 1 (SP1) or Windows Server 2008. By using SSL, SSTP-based VPN connections provide data confidentiality, data integrity, and data authentication.

  • All four tunnel types carry PPP frames on top of the network protocol stack. Therefore, the common features of PPP, such as authentication schemes, Internet Protocol version 4 (IPv4) and Internet Protocol version 6 (IPv6) negotiation, and Network Access Protection (NAP), remain the same for the three tunnel types.

Additional references