Appendix D: Testing the Windows Server 2003 Security Guide

Published: December 31, 2003   |   Updated: April 26, 2006

Overview

The Windows Server 2003 Security Guide is designed to provide proven and repeatable configuration guidance to secure computers that run Microsoft® Windows Server™ 2003 with Service Pack 1 (SP1) in a variety of environments.

The Windows Server 2003 Security Guide was tested in a lab environment to ensure that the guidance works as expected. The documentation was checked for consistency and all recommended procedures were tested by the Windows Server 2003 Security Guide test team. Tests were performed to verify functionality, but also to help reduce the amount of resources that are needed by those who use the guidance to build and test their own implementations.

Scope

The Windows Server 2003 Security Guide was tested in a lab that simulated three different security environments—Legacy Client (LC), Enterprise Client (EC), and Specialized Security – Limited Functionality (SSLF). These environments are described in Chapter 1, "Introduction to the Windows Server 2003 Security Guide." Tests were conducted based on the criteria that are described in the following "Test Objectives" section.

A vulnerability assessment of the test lab environment that was used to secure the Windows Server 2003 Security Guide solution was out of scope for the test team.

Test Objectives

The Windows Server 2003 Security Guide test team was guided by the following test objectives:

  • Validate the recommended changes in security settings for the three security levels that are defined in the guide. Reasons for these changes include:
    • Changes caused by the release of SP1 for Windows Server 2003.
    • Use of the new Security Configuration Wizard (SCW) tool that became available in SP1 and new features such as Windows Firewall.
    • Internal and external feedback that was received about the previous version of the guide.
  • Ensure that the security settings and configuration changes that are recommended in the guide meet the requirements of the LC, EC, and SSLF environments.
  • Ensure that hardened domain member servers are able to successfully perform their role tasks.
  • Ensure that communication between the client computers and the domain controllers is not negatively affected.
  • Verify that all prescriptive guidance is clear, complete, and technically correct.

Finally, the guidance should be repeatable and reliably usable by a Microsoft Certified Systems Engineer (MSCE) with two years of experience.

Test Environment

The test lab networks that were developed to test this guide were similar to those that were used in the previous version of the guide. Three separate but similar networks were developed, one for each of the defined environments.

Each test network consisted of a Windows Server 2003 with SP1 Active Directory® directory service forest, computers for infrastructure server roles that provided domain controller, DNS, WINS and DHCP services, and other computers for application server roles that provided file, print, and Web services. The EC network also included computers for the Certificate Services and IAS server roles, and the Bastion Host (BH) server role was included in the SSLF network. Also, the EC and SSLF networks included Microsoft Operations Manager (MOM) 2005 and Systems Management Server (SMS) 2003 to manage and monitor the domain member servers and client computers. These networks also included Microsoft Exchange Server 2003 for e-mail service.

The client computers in the different networks used Windows XP Professional with SP2 and Windows 2000 Professional with SP4. The LC network also included client computers that ran the Windows 98 SR2 and Windows NT® 4.0 workstation with SP6a operating systems.

The following diagram shows the test lab network that was developed for the EC environment.

 

Figure D.1 Logical diagram of the test lab network for the EC environment

Figure D.1 Logical diagram of the test lab network for the EC environment

See full-sized image

 

To verify replication scenarios between hardened domain controllers, the Active Directory forest consisted of two sites. One site was the main office site with an empty root domain and a child domain that consisted of the previously mentioned server and client computers. The second site consisted merely of a single second domain controller of the child domain.

The following diagram shows the test lab network that was developed for the SSLF environment.

 

Figure D.2 Logical diagram of the test lab network for SSLF environment

Figure D.2 Logical diagram of the test lab network for SSLF environment

See full-sized image

 

Testing Methodology

This section describes the procedures that were followed to test the Windows Server 2003 Security Guide.

The test team established a lab that incorporated the three networks that are described in the previous section. A quick proof of concept (POC) test pass and then two more robust test cycles were executed. During each pass the team strove to stabilize the solution.

A test cycle was defined as a sequence of the following phases:

  1. Security Configuration Build phase
    • Manual configuration phase
    • Group Policy configuration phase
  2. Test Execution phase

The details of each phase are provided in the following "Phases in a Test Pass" section. The "Test Preparation Phase" section describes the steps that were performed to ensure that the lab environment was free of any issues that could cause a misinterpretation of the actual test results after the three environments were hardened through the first two incremental build phases. It is also referred to as the “baseline” state.

Phases in a Test Pass

The test pass phases are described in the following subsections. Any critical issues that were found during the build phase were identified as bugs and resolved in that phase before the test team moved to the test execution phase. This method ensured that correct security configuration was implemented in the network and validated the accuracy of the test results that were obtained.

Test Preparation Phase

This phase set up the baseline configuration to which the solution is applied during the Security Configuration Build phase. The following steps were performed for each of the three environments—LC, EC, and SSLF:

To complete the test preparation phase

  1. Network the computers as illustrated in the network diagram and install the appropriate versions of the Windows operating system on all server and client computers.
  2. Create and configure the domain, domain controllers, and the two sites.
  3. Join and configure each member server and the management servers. Also, join the client computers to the domain.
  4. Execute basic verification tests for each server role to confirm proper network and application configuration.
  5. Check the event log of each member server in the network to ensure that there are no application or system level errors.
  6. Ensure client computer accessibility to the services that are provided by the domain controller and member servers (DNS, DHCP, CA, file, print, Web and e-mail). Check the event logs on the client computers to ensure that there are no errors.
  7. Ensure that all required applications, services, and agents are installed on each domain member. For example, verify that the MOM agent is installed on all the servers that will be managed by the MOM server.
  8. After the previous steps are completed, create an image backup of each computer. These backup images are used to "roll back" the network to the baseline configuration before a new test pass is started.

Security Configuration Build Phase

The objective of this phase was to follow the procedures in the guide to configure the domain, domain controllers, and member servers to a more secure level than the baseline configuration.

Manual Configuration Phase

This phase is often the first security build phase. The manual hardening recommendations that were provided in each chapter were implemented during this phase.

Note: Some of these steps may be applicable for your network and some may not. Review each procedure carefully to understand its impact on your network.

To perform the manual configuration phase

  1. Use the Microsoft Management Console (MMC) Computer Management snap-in to perform the prescribed policy setting changes (such as the local administrator account and password) on each member computer. Complete the following steps to secure the domain accounts:
    1. Ensure that the built-in local Administrator account has a complex password, has been renamed, and has had its default account description removed.
    2. Rename the Guest accounts on the host and disable them.
    3. Incorporate any additional recommendations from the guide about how to secure the domain accounts.
  2. Add any unique security groups or accounts to the user rights settings as described in the chapters.
  3. Perform all other applicable manual hardening procedures as prescribed in each chapter. For example, enable Manual Memory Dumps and Error reporting configuration.

Group Policy Configuration Phase

The purpose of this phase is to create and apply the Group Policy objects (GPOs) at the domain and organizational unit (OU) levels. GPOs are applied to the different OUs based on the recommendations in Chapter 2, "Windows Server 2003 Hardening Mechanisms."

Service Pack 1 for Windows Server 2003 introduced some new tools and features that caused the Group Policy implementation design to change from its previous version.

SCW is an attack-surface reduction tool that is used to create the required set of security policies for each of the server roles that are discussed in this guide. The availability of SCW caused the following two significant changes for the Group Policy Configuration Phase:

  • IPsec filters that were provided with the previous version of this guide were replaced with Windows Firewall port configurations that were created with SCW.
  • Security templates that are included with the guide are to be used in conjunction with SCW to create XML security template files. These templates are then converted to corresponding GPOs using the Scwcmd command-line tool.

The following steps were repeated for each of the three security environments:

To create Group Policy objects

  1. Ensure that all required applications, services, and agents were installed on each domain member in the baseline network. For example, ensure that the MOM agent was installed on all the domain member servers that will be managed by MOM.
  2. Use the MMC Active Directory Users and Computers snap-in to create the described OU structure.
  3. Create the Domain Policy GPO with the .inf security template. This step does not require the use of SCW.
  4. Use the SCW tool to create XML–based security templates for each server role that is described in the guide. Prescriptive steps are described in Chapter 2, "Windows Server 2003 Hardening Mechanisms" and each individual server role chapter. When you perform this step, include the appropriate .inf security template for the server role. The template files are included with the downloadable version of this guide.
  5. Use the Scwcmd command-line tool to convert the XML security templates that were created in the previous step to GPOs.
  6. Repeat step 4 on the Bastion Host server to create the Bastion Host XML security template and then use SCW again to convert and apply it to the Local GPO.

After the GPOs are successfully created, compare the settings with the guidance in the chapters to identify any incorrect configurations.

At this stage, all the domain member servers reside in the Computers OU. These servers are then moved to their respective OUs under the Member Server OU.

The next task (detailed in the following procedure) is to apply each of these GPOs to the respective OUs. The Group Policy Management Console (GPMC) tool was used to link the GPO with the OU. The Domain Controller Policy GPO was linked last.

The following steps were performed to complete the rest of the Security Configuration Build phase:

To apply Group Policy objects

  1. Link the Domain Policy GPO to the domain object.Note: If default GPO links are already present or if there are multiple GPOs, you might need to elevate the GPO links in the priority list.
  2. Use the Group Policy Management Console tool to link the Member Server Baseline Policy GPO to the Member Servers OU. (You can also perform this step with the MMC Active Directory Users and Computers snap-in.)
  3. Link each individual server role GPO to the appropriate server role OU.
  4. Link the Domain Controller Policy GPO to the Domain Controller OU.
  5. To ensure application of the latest Group Policy settings, execute gpudpate /force at a command prompt on all domain controllers. Then restart all the domain controllers one at a time, starting with the primary domain controller. Allow sufficient time for Active Directory to replicate the changes between the sites.Important: It is very important to restart the domain controllers after you apply the Domain Controllers Policy GPO. If you do not perform this step you may see replication errors in the Directory Service folder or Userenv errors in the Application folder of Event Viewer.
  6. Repeat step 5 on all of the domain member servers.
  7. Check Event Viewer for any errors. Review the error logs to troubleshoot and resolve any failures.
  8. On the Bastion Host server, use the SCW tool to apply the Bastion Host XML security template on the Local GPO of the server.

Verifying Group Policy Download on the Member Server Computers

The previous procedures created GPOs and applied them to OUs to configure the computers in those OUs. Complete the following steps to confirm the successful download of Group Policy from domain controllers to member server computers. (It is assumed that the member server computers were restarted after the GPO was linked to the OU.)

To verify Group Policy download on a member server computer

  1. Log on to the member server computer.
  2. Click Start, Run, type rsop.msc, and press ENTER.
  3. In the Resultant Set of Policy console, expand Console Root and browse to Computer Configuration.
  4. Right-click Computer Configuration and click Properties.The list of GPOs will display in the Computer Configuration Properties panel. The GPO that was applied to the OU should be available in the list, and there should be no errors associated with it.

Test Execution Phase

This phase executes the test cases that were developed by the test team. The test execution phase seeks to identify the following:

  • Any potential application, security, or system failure events that are caused by processes that were used to harden the domain, domain controllers, member servers, or Bastion Host server.
  • Lost availability of a service or functionality that is caused by changes to the security configuration of the servers in the network.
  • Technical inaccuracies between what is documented in the chapters and the physical implementation in the test lab.

The test team executed the set of test cases that are included in \Windows Server 2003 Security Guide Tools and Templates\Test Tools folder. (The tools and templates are included with the downloadable version of this guide.) These tests were executed on each of the three separate networks except for those that tested components that were only available in one network—such as Certificate Services, which was only available in the EC environment. In addition to these test cases, manual testing was performed at various time—for example, to periodically check Event Viewer logs or to verify any specific issues that were discovered in the previous version of the guide. All issues that were found were logged in a database and triaged with members of the development team until they were resolved.

More detailed information about the different types of tests that were performed is provided in the next section.

Types of Tests

The test team performed the following types of tests during the test phases to ensure that the secured domain, domain controllers, and member servers did not experience any significant loss of functionality. You may want to refer to the Excel workbooks in the \Windows Server 2003 Security Guide Tools and Templates\Test Tools folder that is included in the download for this guide, which contain the complete list of test cases that were executed for domain–based as well as stand-alone servers that run Windows Server 2003 with SP1. Details such as test scenarios, execution steps, and expected results are also provided.

These tests were executed multiple times. More importantly, they were executed before and after the security settings that are described in this guide were implemented. This approach helped the test team to identify potential errors and any variations in functionality for the listed server roles.

Client Side Tests

These test cases were executed on the client computers in the network. The main purpose of these tests was to ensure that domain services (such as authentication, access rights, name resolution, and so on) and application based services (such as File, Print, and Web) are available to the client computers after the network servers are hardened. For the LC environment, these tests ensured that those client computers that run Windows NT 4.0 SP6a and Windows 98 were able to authenticate with the Windows Server 2003 Active Directory domain.

Documentation Build Tests

These tests validate that the statements, procedures, and functions that are documented in the implementation guidance are accurate, unambiguous, and complete. No separate test cases are listed for these tests.

Script Tests

Some of the client test scenarios were scripted in VBScript. These test cases are primarily concerned with proper functionality of Windows XP client computers that use network–based services, such as domain logon, password change, and print server access. The VBScript files for these test cases are available in the \Windows Server 2003 Security Guide Tools and Templates\Test Tools folder that is included in the downloadable version of this guide.

Server Side Tests

These test cases were developed to verify functionality and the effect of the build procedures on Windows Server 2003 with SP1 servers that were secured with the recommendations in this guide. All the server roles that are described in this guide were tested. The additional server roles that were included in the test network, such as Exchange, MOM, and SMS, were also tested.

Pass and Fail Criteria

Before tests were performed, the following criteria were defined to ensure defect prevention and bug resolution:

  • All test cases must pass with expected results as described in the individual test case spreadsheets.
  • A test case is considered to have passed if the actual result matched the expected result that is documented for the case. If the actual result does not match the expected result, it was treated as a failed test case, a bug was created, and a severity score was assigned.
  • If a test case failed, it was not assumed that the solution guidance was necessarily defective. For example, misinterpretation of product documentation, incomplete documentation, or inaccurate documentation could cause failures. Each failure was analyzed to discover its cause based on actual results and the results that were described in project documentation. Failures were also escalated to the appropriate owners of the respective Microsoft products.

Release Criteria

The primary release criterion for the Windows Server 2003 Security Guide was related to the severity of bugs that were still open. However, other issues that were not being tracked through bugs were also discussed. The criteria for release are:

  • No bugs are open with severity levels 1 and 2.
  • All open bugs are triaged by the leadership team, and their impacts are fully understood.
  • Solution guides are free of comments and revision marks.
  • The solution successfully passes all test cases in the test lab environment.
  • Solution contents have no conflicting statements.

Bug Classification

The bug severity scale is described in the following table. The scale is from 1 to 4, with 1 as the highest severity and 4 as the lowest severity.

Table D.1 Bug Severity Classification

Severity Most common types Conditions required

1

– Bug blocked build or further testing.– Bug caused unexpected user accessibility.– Steps defined in the documentation were not clear.– Results or behavior of a function or process contradicts expected results (as documented in functional specification).– Major mismatch between the security template files and the functional specification.

– Solution did not work.– User could not begin to use significant parts of the computer or network.– User had access privileges that should not be allowed.– User access was blocked to certain server(s) that should be allowed.– Expected results were not achieved.– Testing cannot proceed without being addressed.

2

– Steps defined in the guide are not clear.– Documented functionality is missing (in this case, test was blocked).– Documentation is missing or inadequate.– Inconsistency between security template files and content in the guide, but security template file is in sync with functional specification.

– User had no simple workaround to amend the situation.– User could not easily figure out a workaround.– Primary business requirements could not be met by the computer or network.

3

– Documented format issue.– Minor documentation errors and inaccuracies.– Text misspellings.

– User has a simple workaround to mend situation.– User can easily figure out workaround.– Bug does not cause a bad user experience.– Primary business requirements are still functional.

4

– Suggestions.– Future enhancements.

– Clearly not related to this version.

Summary

This appendix enables an organization that uses the Windows Server 2003 Security Guide to understand the procedures and steps that were used to test the implementation of the solution in a test lab environment. The actual experience of the Windows Server 2003 Security Guide test team is captured in this appendix, which includes descriptions of the test environment, types of tests, the release criteria, and bug classification details.

All of the test cases that were executed by the test team passed with the expected results. The test team confirmed that the requisite functionality was available after the recommendations from the Windows Server 2003 Security Guide for the defined environments were applied.

This accelerator is part of a larger series of tools and guidance from Solution Accelerators.

Download

Get the Windows Server 2003 Security Guide

Solution Accelerator Notifications

Sign up to stay informed

Feedback

Send us your comments or suggestions