Designing an IPSec Policy

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

You need to create a set of IPSec policies that matches the needs of your environment. Organizations that have consistent security needs and a more simple network structure can create fewer and less complex policies to meet their goals. Other organizations with more stringent security needs and more complex environments require a greater number of policies and potentially more rules in their policies.

Use the steps shown in Figure 6.9 to help you decide how restrictively secure you want to make your IPSec policies.

Figure 6.9   Designing IPSec Policies

When designing an IPSec policy, you need to decide whether to apply an IPSec policy to protect traffic just for specific paths (between specific IP addresses) or to provide general protection for all traffic sent to and from a specific computer. If you want to use IPSec to provide general protection for all traffic sent to and from a specific computer, configure a policy that is based on blocking all traffic and then permitting exceptions, or permitting all traffic and then blocking particular ports. Blocking all traffic provides greater security, but this approach also requires a detailed analysis of the computer’s communications, to ensure that required traffic is not blocked.

Consider a simple scenario that requires a simple IPSec policy strategy. All computers belong to a single Active Directory domain. The environment is composed entirely of servers running Windows Server 2003 and clients running Windows XP. All servers (except domain controllers) request confidentiality by using ESP, and all clients support IPSec. In such a case, only two policies are required: one for clients, and one for member servers. These policies can be applied by using Group Policy at the OU level in Active Directory. Because a different IPSec policy is applied to servers, the server computer accounts are grouped into their own OU. A GPO is created for the OU, and then the IPSec server policy is assigned to that GPO.

Because of the operating systems that the computers run and because all of the computers are domain members, both the IPSec server and client policies require that Kerberos be used for mutual authentication. To ensure that domain controllers do not receive any IPSec policy, the predefined domain security group for domain controllers is denied Read access to the GPO that assigns IPSec client policy at the domain level.

Conversely, many more policies can be required in a more complicated scenario. A single IPSec policy can contain many rules, each rule tailored for a specific type of traffic. Many different IPSec policies might need to be designed, to meet the security requirements different computers. Factors that can increase the number of policies required in your environment include:

• Computer roles. Different servers performing in different roles require different types of security, including some combination of packet filtering, confidentiality, integrity, and tunneling. Meeting the security needs of an environment that includes many servers with different roles and requirements can require hundreds of different configurations per server. Furthermore, servers might serve multiple roles, which complicates role-based configurations.

• Sensitivity of data sent over the network. Different levels of encryption are required to accommodate different security needs. For example, 3DES, which provides greater security than DES, can be used to protect transmission of personnel files stored on a file server.

• Computer operating systems. Some operating systems automatically support IPSec transport mode, but some require special client software to be installed. Depending on their operating system, computers might receive IPSec policy through different means: some by Group Policy, others only through locally installed policies. Additionally, Windows Server 2003 IPSec includes several new features that are not supported by Windows 2000 or Windows XP.

• Domain memberships. Some clients and servers belong to domains in different forests, some to UNIX realms, and some are members of workgroups. The trusts and authentication methods in place between such forests, realms, and workgroups affect which IPSec configurations are available.

• Domain relationships. As above, what IPSec configurations are available and what authentication is possible depends on whether domains are in the same forest or different forests, and on whether those forests trust each other.

If you have a complex environment, use IPSec only where it is truly needed. Although you want to provide an appropriate level of security, use as few policies as possible to minimize the complexity of your system. A simpler system is less likely to produce problems and is also easier to troubleshoot if it does.

After identifying what network segments, communications, and computers you want to secure by using IPSec, you need to specify the settings and rules that make up an IPSec policy. The settings and rules you specify also determine how strictly security is enforced by your IPSec policies. Not only must you put policies in place to meet your security requirements, but you must also ensure that computers on your network have compatible policies that can allow them to negotiate SAs.

General IPSec Policy Settings

General IPSec policy settings must be specified whether you want the policy to provide packet filtering or end-to-end security. Make sure to manage all IPSec policies in a controlled way from design, through testing, and into production. To ensure proper management, include version numbers in policy names. General IPSec policy settings are shown in Table 6.4.

Table 6.4   General IPSec Policy Settings

Note

• Computers running Windows Server 2003 and Windows XP support the 3DES and DES algorithms and do not require installation of any additional components. However, computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use 3DES. If a computer running Windows 2000 is assigned a policy that uses 3DES encryption, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the security method defaults to the weaker DES algorithm. To ensure at least some level of privacy for communication, make sure to allow DES as a fallback option whenever a 3DES setting is applied to a group of computers in case some of them cannot support 3DES.

IPSec Rules

IPSec rules determine which traffic is affected by an IPSec policy and which actions take place when that type of traffic is encountered. Table 6.5 describes the contents of IPSec rules that two computers use to establish a secure, authenticated channel.

Table 6.5   Settings of IPSec Rules

For more information about IPSec policy rules, see "IPSec Policy Rules" in Help and Support Center for Windows Server 2003.