Appendix B: Testing the Windows XP Security Guide

Updated: April 13, 2006

Introduction

The function of the Windows XP Security Guide is to provide proven and repeatable configuration guidance to secure computers that run Microsoft® Windows® XP Professional with Service Pack 2 (SP2) in a variety of environments.

The Windows XP Security Guide was tested in a lab environment to ensure that the guidance works as expected. The documentation was checked for consistency and all recommended procedures were tested by the Windows XP Security Guide test team. Tests were performed to verify functionality, but also to help users of the guidance to reduce the amount of resources that are needed to build and test their own implementations of the solution.

Scope

The Windows XP Security Guide was tested in a lab environment for two different security environments—Enterprise Client (EC) and Specialized Security – Limited Functionality (SSLF). These environments are described in Chapter 2, "Configuring the Active Directory Domain Infrastructure." Tests were conducted based on the criteria that are described in the following "Test Objectives" section.

A vulnerability assessment of the test lab environment that was used to secure the Windows XP Security Guide solution was out of scope for the test team. Penetration testing was performed by partners.

Test Objectives

The Windows XP Security Guide test team was guided by the following test objectives:

  • Ensure that the prescriptive configuration and policy settings for Windows XP Professional with SP2 interoperate correctly and as expected in a Windows Server™ 2003–based domain network for the two different security environments.
  • Ensure that Windows XP Professional SP2 client computers are able to perform the basic tasks and applications that are listed in the included test cases.
  • Verify that all prescriptive guidance in version 2.1 of the Windows XP Security Guide is clear, complete and technically correct.
  • Verify that the security templates work as expected on the Windows XP Professional SP2 client operating system.
  • Verify that the Administrative Templates and Software Restriction Policy recommendations work as expected on the Windows XP Professional SP2 client operating system.

Finally, the guidance should be repeatable and reliably usable by a Microsoft Certified Systems Engineer (MSCE) with two years of experience.

Test Environment

The test environment consisted of a Windows Server 2003 SP1 Active Directory® directory service, computers for infrastructure server roles that provided domain controller, DNS, and DHCP services, and other computers for application server roles that provided file, print, Web, CA, and Microsoft Exchange 2003 e-mail services. The desktop and laptop client computers in the domain used Windows XP Professional with SP2.

The network also contained two client computers that used Windows XP Professional with SP2 in workgroup mode that were used to test stand-alone security templates. Laptop computers in the domain network were reused to test stand-alone laptop security templates. The following figure illustrates the test network.

 

Figure B.1 The network that was used to test the Windows XP Security Guide in domain and stand-alone mode

Figure B.1 The network that was used to test the Windows XP Security Guide in domain and stand-alone mode

See full-sized image

 

The network in the following figure was developed to test the legacy templates that are included with this guide.

 

Figure B.2 The network that was used to test the legacy security templates that are included with this guide

Figure B.2 The network that was used to test the legacy security templates that are included with this guide

See full-sized image

 

Testing Methodology

This section describes the procedures that were followed to test the Windows XP Security Guide.

The test team established a lab that incorporated the networks that are illustrated in the previous section. The test team executed a quick proof of concept (POC) test pass and then two more robust test cycles. During each pass the team strove to stabilize the solution.

A test cycle was defined as a sequence of the following two incremental build phases:

  1. Manual computer configuration phase
  2. Group/Local Policy configuration phase

The details of each phase are provided in the following "Phases in a Test Pass" section. The "Test Preparation Phase" section describes the steps that were performed to ensure that the lab environment was free of any issues that could cause a misinterpretation of the actual test results after both of the environment scenarios were hardened through the two incremental build phases.

In each test pass, different sets of test cases were executed. These tests are explained in the "Types of Tests" section later in this appendix.

Phases in a Test Pass

This solution was tested in the phases that are described in the following subsections. Any critical issues that were found in a build phase were raised as bugs and resolved in that phase before the test team moved to the next incremental phase. This method ensured that critical issues were resolved quickly. It also minimized the need for resources that would be needed to debug issues that were found in later phases.

Test Preparation Phase

This phase set up the baseline network to which the solution was applied. It consisted of the following steps.

To perform the test preparation phase

  1. Network the computers as illustrated in the network diagram and install the appropriate versions of the Windows operating system on all server and client computers.
  2. Create and configure domain controllers, domains, and each server role. Join the Windows XP Professional with SP2 client computers to the domain.
  3. Install user applications on each of the Windows XP Professional with SP2 client computers.
  4. Execute basic verification tests to confirm proper network configuration. Ensure client computer accessibility to the services that are provided by the domain controller and member servers (DNS, DHCP, CA, file, print, Web and e-mail).
  5. Execute the installed applications to verify that there are no installation problems and that all the applications run properly.
  6. Check the event log to ensure that there are no errors.
  7. After the previous steps are completed, create an image backup of each computer. These backup images are used to "roll back" the network to the default state before a new test pass is started.

Manual Configuration Phase

This phase is often the first security build phase. It consists of the following build procedure.

To perform the manual configuration phase

  1. The Microsoft Management Console (MMC) Computer Management snap-in is used to perform the prescribed policy setting changes, such as the local administrator account and password on each member computer. Complete the following steps to secure the domain accounts (Guest and Administrator accounts):
    1. Disable the Guest account.
    2. Ensure that the built-in Administrator account has a complex password, has been renamed, and has had its default account description removed.
    3. Incorporate any additional recommendations from the guide about how to secure the domain accounts.
  2. Perform all other applicable manual hardening procedures as prescribed in each chapter of the guide.
  3. For stand-alone Windows XP client computers, manually create a secure database.

Group/Local Policy Configuration Phase

In this phase, the Group Policy objects (GPOs) are applied at the domain and organizational unit (OU) levels. GPOs are applied to the different OUs based on the recommendations in Chapter 2, "Configuring the Active Directory Domain Infrastructure." For stand-alone Windows XP client computers, local policy is configured. This phase consists of the following steps.

To perform the Group/Local Policy configuration phase

  1. Create the described OU structure to support Group Policy recommendations that are made in the guide.
  2. Move the Windows XP desktop and laptop client computers to the appropriate OUs.
  3. Identify the domain users and move them to the appropriate OUs so that you can apply the Administrative Templates.
  4. Add a new GPO link for each OU.Note: You might need to elevate the GPO links in the priority list where default GPO links are already present.
  5. Import the security template that was included with the guide into the GPO.
  6. For each environment scenario in the different chapters, apply the appropriate Group Policy on each OU.

Test Execution Details

Chapters 2 through 6 of the Windows XP Security Guide provide instructions for applying the security recommendations to the domain, Windows XP desktop computers, Windows XP laptop computers, and Windows XP stand-alone client computers for the Enterprise Client (EC) and Specialized Security – Limited Functionality (SSLF) environments that are defined in the guide. These recommendations are accompanied by a Microsoft Excel® workbook, security templates, Administrative Templates and automated scripts. The automated scripts are used to import templates into the local GPO on the secure stand-alone client computers. This section explains how the recommendations were implemented and tested.

Chapter 2: Configuring the Active Directory Domain Infrastructure

Complete the following procedures to test this chapter.

To verify the baseline network

  • Complete the basic verification test cases to ensure that the image backups function properly. A successful completion of these test cases confirms that the entry criteria are met.

To start the manual configuration phase

  1. Synchronize the time of all the domain member servers and the Windows XP client computers with the domain controller.
  2. Disable the Guest account.
  3. Rename the Administrator and Guest accounts.
  4. Change the Administrator password.

To implement the OU structure configuration

  1. In the corp.woodgrovebank.com domain, create an OU with the name "Department OU."
  2. Create two sub-OUs in the Department OU. Name them "Windows XP OU" and "Secured XP Users OU."
  3. In the Windows XP OU, create the following four sub-OUs:
    • EC Desktop OU
    • EC Laptop OU
    • SSLF Desktop OU
    • SSLF Laptop OU
  4. Move the Windows XP computers that are assigned to each security environment to their respective OUs.
  5. Move the domain users that will log on to the Windows XP client computers to the Secured XP Users OU.
  6. Create and link a new GPO with the name "Domain Policy" on the corp.woodgrovebank.com object. Click Up on the Group Policy tab of domain object in the MMC Active Directory Users and Computers snap-in to ensure the highest priority for the new GPO, and then import the appropriate security template (SSLF-domain.inf or EC-domain.inf) into the GPO.
  7. Execute gpupdate /force on the domain controller to download the latest Group Policy settings.

Chapter 3: Security Settings for Windows XP Clients

This chapter describes the primary settings that are configured through Group Policy in a Windows Server 2003 domain. The chapter prescribes policy settings for the two defined security environments to ensure that Windows XP with SP2 desktops and laptops are secure.

To configure the security template settings

  1. Execute the base deployment tests to verify that all the recommendations in the guide are appropriate for your environment. Review the recommended policy settings. Modify the settings in the security templates as needed before you proceed to deploy them.
  2. Link new GPOs to each of the two Desktop OUs. For the Enterprise Client environment, import the EC-desktop.inf security template into the GPO. For the Specialized Security – Limited Functionality environment, import the SSLF-desktop.inf security template into the GPO.
  3. Link new GPOs to each of the two Laptop OUs. For the Enterprise Client environment, import the EC-Laptop.inf security template into the GPO. For the Specialized Security – Limited Functionality environment, import the SSLF-Laptop.inf security template into the GPO.
  4. Log on to a Windows XP client computer and execute the gpupdate /force command. Then restart the computer to ensure that the latest Group Policy settings are downloaded.
  5. Run the tests that are listed later in this document.

Chapter 4: Administrative Templates for Windows XP

This chapter describes how to configure and apply additional policy settings on computers that run Microsoft Windows XP with SP2 by using Administrative Templates.

To configure the Administrative Template settings

  1. Execute the base deployment tests to verify that all the recommendations in the guide are appropriate for your environment. Review the recommended policy settings. Modify the settings in the Administrative Templates as needed before you proceed to deploy them.
  2. Create four new GPOs, one for each of the four types of Windows XP client computers. Because there is some variation in the policy settings for desktops and laptops, it is suggested that you create separate GPOs for each.
    • EC Desktop Administrative Template Policy
    • EC Laptop Administrative Template Policy
    • SSLF Desktop Administrative Template Policy
    • SSLF Laptop Administrative Template Policy
  3. In the Administrative Templates, configure the computer configuration settings and the user configuration settings for each of the GPOs according to the guidance that is provided in Chapter 4, "Administrative Templates for Windows XP."
  4. Link the GPOs to their respective OUs.
  5. Log on to a Windows XP client computer and execute the gpupdate /force command. Then restart the computer to ensure that the latest Group Policy settings are downloaded.
  6. Run the tests that are listed later in this appendix.

Chapter 5: Securing Stand-Alone Windows XP Clients

This chapter describes the primary policy settings that are set through local computer policy. The prescribed setting values will help ensure that stand-alone desktops and laptops in the organization that run Windows XP with SP2 are secure.

To configure security settings on stand-alone Windows XP clients

  1. Execute the base deployment tests to confirm that all recommendations in the guide are appropriate for your environment.
  2. Use the MMC Security Configuration and Analysis snap-in to create a security database. This database will be used to write to local policy. Step-by-step guidance is provided in Chapter 5, "Securing Stand-Alone Windows XP Clients."
  3. Use the Security Configuration and Analysis snap-in to apply the policy settings that are included in the stand-alone security template files. Step-by-step guidance is provided in Chapter 5, "Securing Stand-Alone Windows XP Clients." It is important to use the Security Configuration and Analysis snap-in, because system services policy settings cannot be applied with the Local Computer Policy snap-in.
  4. Run the appropriate automated script (included with this guide) to import the security templates.
  5. Run the tests that are listed later in this appendix.

Chapter 6: Software Restriction Policy for Windows XP Clients

This chapter allows administrators to identify and control the software that runs in their domain. The tool that is used to accomplish this control is a policy-driven mechanism called software restriction policy.

To configure software restriction policy

  1. Execute the base deployment tests to confirm that all recommendations in the guide are appropriate for your environment.
  2. Locate the OU that was created for the Windows XP desktops and laptops. For stand-alone client computers, the policy settings are located in the local security policy. Create a new GPO for the Windows XP OU. Remember, this new GPO is only used for the software restriction policy.
  3. Configure the software restriction policy as follows:
    1. Create a default software restriction policy.
    2. Set up the path rules.
    3. Set the policy options, such as enforcement, designated file types, and trusted publishers according to the prescriptions that are provided.
  4. Review the policy settings and then reset the default policy setting to Disallowed.
  5. Log on to a Windows XP client computer and execute the gpupdate /force command. Then restart the computer to ensure that the latest Group Policy settings are downloaded.
  6. Run the tests that are listed later in this appendix.

Verifying Group Policy Download on the XP Client

In the previous sections, GPOs were applied to OUs, which then applied the GPOs to the computers in the OUs. Complete the following steps to confirm the successful download of Group Policy from the domain controller to a Windows XP client computer. It is assumed that the client computer was restarted after the GPO was linked to the OU.

To verify Group Policy download on a Windows XP client computer

  1. Log on to the Windows XP client computer.
  2. Click Start, Run, type rsop.msc, and press ENTER.
  3. In the Resultant Set of Policy console, expand Console Root and browse to Computer Configuration.
  4. Right-click Computer Configuration and click Properties.The list of GPOs will display in the Computer Configuration Properties panel. The GPO that was applied to the OU should be available in the list, and there should be no errors associated with it.
  5. Verify the Administrative Templates policy settings.Only the settings that are configured in the Administrative Template GPO should be visible in the respective Administrative Templates folder tree under Computer Configuration or User Configuration.

Types of Tests

The test team performed the following types of tests during the testing phases to ensure that the secured Windows XP client computers are able to perform basic tasks without significant loss of functionality. You may want to refer to the Excel workbook "Windows XP Security Guide Test Cases.xls," which is in the \Windows XP Security Guide Tools and Templates\Test Tools folder that is included in the download for this guide. This workbook file contains the complete list of test cases that were executed for domain–based XP client computers and stand-alone XP client computers, as well as details such as test scenarios, execution steps, and expected results.

Application Tests

These tests check whether user applications that are installed on the Windows XP client computers (such as the Office 2003 application suite, Windows Media® Player, and a few more) work properly. For more details about the test cases, refer to the Microsoft Excel workbook Windows XP Security Guide Test Cases.xls that is included with this guide.

Automated Script Tests

Some of the test case scenarios were scripted in VBScript. These test cases are primarily concerned with proper functionality of Windows XP client computers that use network–based services, such as domain logon, password change, and print server access. The VBScript files for these test cases are available in the \Windows XP Security Guide Tools and Templates\Test Tools folder that is included in the download for this guide.

Basic Verification Tests

These test cases are a subset of the Application, Automated Script and Internet tests. They are basic tests that cover a variety of different scenarios, such as the ability to run applications that are installed on the client, client-server communication tests, the ability to access the Internet and download patches, and tests that monitor errors on the host. These test cases are also executed when you establish a baseline for the network during the Test Preparation phase.

Documentation Build Tests

These tests validate that the statements, procedures, and functions that are documented in the implementation guidance are accurate, unambiguous, and complete. No separate test cases are listed for these tests.

Functional Tests

These tests are designed to verify that the system that was built from the build guidance works correctly and as expected. They verify the functionality, health, and effect of the build procedures on the desktop and laptop client computers.

Internet–Based Tests

Today's computer users typically need to access the Internet. These test cases ensure that some of the common day-to-day capabilities (browse to Web sites, use the Windows Messenger service, and download critical updates from the Microsoft Update site) are not affected by the lockdown of the Windows XP client computer.

Pass and Fail Criteria

Before tests were performed, the following criteria were defined to ensure defect prevention and bug resolution:

  • All test cases must pass with expected results as described in the individual test case spreadsheets.
  • A test case is considered to have passed if the actual result matched the expected result that is documented for the case. If the actual result does not match the expected result, it was treated as a failed test case, a bug was created, and a severity score assigned.
  • If a test case failed, it was not assumed that the solution guidance was necessarily defective. For example, misinterpretation of product documentation, incomplete documentation, or inaccurate documentation could cause failures. Each failure was analyzed to discover its cause based on actual results and the results that were described in project documentation. Failures were also escalated to the appropriate owners of the respective Microsoft products.

Release Criteria

The primary release criterion for the Windows XP Security Guide was related to the severity of bugs that were still open. However, other issues that were not being tracked through bugs were also discussed. The criteria for release are:

  • No bugs are open with severity levels 1 and 2.
  • All open bugs are triaged by the leadership team, and their impacts are fully understood.
  • Solution guides are free of comments and revision marks.
  • The solution successfully passes all test cases in the test lab environment.
  • Solution contents have no conflicting statements.

Bug Classification

The bug severity scale is described in the following table. The scale is from 1 to 4, with 1 as the highest severity and 4 as the lowest severity.

Table B.1 Bug Severity Classification

Severity Most common types Conditions required

1

– Bug blocked build or further testing.– Bug caused unexpected user accessibility.– Steps defined in the documentation were not clear.– Results or behavior of a function or process contradicts expected results (as documented in functional specification).– Major mismatch between the security template files and the functional specification.

– Solution did not work.– User could not begin to use significant parts of the system.– User had access privileges that should not be allowed.– User access was blocked to certain server(s) that should be allowed.– Expected results were not achieved.– Testing cannot proceed without being addressed.

2

– Steps defined in the guide are not clear.– Documented functionality is missing (in this case, test was blocked).– Documentation is missing or inadequate.– Inconsistency between security template files and content in the guide, but security template file is in sync with functional specification.

– User had no simple workaround to amend the situation.– User could not easily figure out a workaround.– Primary business requirements could not be met by the system.

3

– Documented format issue.– Minor documentation errors and inaccuracies.– Text misspellings.

– User has a simple workaround to mend situation.– User can easily figure out workaround.– Bug does not cause a bad user experience.– Primary business requirements are still functional.

4

– Suggestions.– Future enhancements.

– Clearly not related to this version.

Summary

This document enables an organization that implements the Windows XP Security Guide to understand the procedures and steps that were used to test the implementation of the solution in a test lab environment. The actual experience of the Windows XP Security Guide test team is captured in this document, which includes descriptions of the test environment, types of tests, the release criteria, and bug classification details.

All of the test cases that were executed by the test team passed with the expected results. The test team confirmed that the requisite functionality was available after the recommendations from the Windows XP Security Guide for the defined environments were applied.

This accelerator is part of a larger series of tools and guidance from Solution Accelerators.

Download

Get the Windows XP Security Guide

Solution Accelerator Notifications

Sign up to stay informed

Feedback

Send us your comments or suggestions