Deploying Windows XP - Application Compatibility

  • Article

Published: January 01, 2002 | Updated: November 23, 2005

By Todd Phillips

Microsoft Corporation

Abstract

This article describes the overall process for testing application compatibility during the deployment of Microsoft Windows XP. It's written for information technology (IT) professionals who are planning the deployment of Windows XP and focuses on how to test applications, what tools are available to assist with testing, and how to resolve incompatibilities.

On This Page

Introduction
Application Compatibility Testing—A Brief Overview
Gathering an Application Inventory
Assembling Your Test Computers
Testing Your Applications
Identifying Resolutions to Compatibility Issues
Summary
Related Links

Introduction

This article discusses the planning steps concerned with application compatibility testing.

Getting Started

If you're in the position of having to plan the deployment of Windows XP to your existing network environment, you need to know the steps for testing your existing applications for compatibility with this new operating system. These steps can also become part of your testing program for the acquisition of new software for your organization. Many of these steps can also be used by independent software vendors (ISVs) as suggestions for designing their own testing procedures.

Topics Covered

Topics covered in the article include:

  • Gathering an Application Inventory

  • Assembling Your Test Computers

  • Testing Your Applications

  • Identifying Resolutions to Compatibility Issues

Application Compatibility Testing—A Brief Overview

This overview briefly discusses what's required to do application compatibility testing and who needs to be involved with each step:

  • Inventory current software. IT staff should gather a list of the current software configurations present in the organization.

  • Build test lab. Testing staff should assemble a group of computers that are representative of the various computer configurations found in the organization.

  • Test current applications. Testers should use AppVerifier while running tests on the applications currently in use within the organization.

  • Test planned applications. The test staff will use AppVerifier on any applications planned for deployment in the organization. Be sure to test the applications in combination with all standard software images.

  • Determine the required compatibility fixes. If any compatibility fixes are required by older applications, testers or developers need to experiment with QFixApp to determine the correct combination of compatibility fixes to resolve issues.

  • Package the compatibility fixes for distribution. IT staff or developers can use the Compatibility Administrator tool to gather all of the compatibility fixes found in the test procedure, for all applications, into a single compatibility database. This file can then be distributed to Windows XP computers that require the application compatibility fixes.

  • Perform a test deployment. IT staff will select a small group of computers, representing each of the common computer configurations in the organization, for a pilot program to deploy Windows XP and the compatibility support that has been developed through the test process. This is the time for testing, IT, and development staff to participate together to resolve any issues that are found.

  • Deploy the software to the organization. Once all issues have been resolved in the pilot program, it is time to perform the final deployment to the organization. All groups within the deployment team (testing, IT, and development) should be available to troubleshoot any issues that may arise during this phase.

Gathering an Application Inventory

This section discusses how to monitor and control the applications installed on client computers.

Monitoring and Controlling Applications

A difficult task in network administration is monitoring and controlling the applications installed on client computers. While users often resent being told what software they are permitted to have installed on their workstations, without such control an effective deployment is highly unlikely.

In an Active Directory™ network, the software configuration of client computers can be controlled through Group Policy. If this has not been done, you can expect to find a wide variety of applications installed on the workstations you intend to upgrade to Windows XP.

Whatever method you finally decide on, you will probably have to settle on several software images. Unless the network you are planning is very small, there will be several different software configurations present because different people will have different tasks. When developing an inventory of the present applications keep this fact in mind and try to identify the common configuration needs within the network.

Automating a Software Inventory

An ideal situation would be one where gathering an inventory of hardware and software is an automatic process.

Microsoft's Systems Management Server (SMS) is a great method for automating the process of gathering hardware and software inventory information over a long period of time. This method enables you to check each computer as it logs onto the network and detect when non-standard software has been installed. SMS can be used to distribute entire applications—or even software updates as they become available.

Adding SMS to your existing network may not be feasible, but if it fits into your plan or is already installed, SMS can cut down on the amount of work needed to create a software inventory. Without a network management tool like SMS, you are left with more manual methods of gathering the inventory information. Care should be taken to clearly identify applications that are required for daily operation. You should also identify applications that, while not required, may be beneficial to your users.

Without an automated system for gathering software inventory information, consider using a survey form that can be distributed to users in your environment. Allow enough time for this process to be completed, and make sure there is management endorsement to ensure that the survey actually gets completed.

Enforcing Standard Software Images

Preventing users from installing shareware or personal programs can be difficult. Group Policy can be used to distribute the standard groups of software that will be used in your network. This makes it easy for users to find and install the applications that they will need to perform their normal tasks.

Group Policy can be used to remove applications that are no longer supported in your network environment. It can even install or uninstall applications based on a person's group membership or location. Group Policy makes a powerful addition to an administrator's toolbox for managing standard software.

To achieve a very high level of standardization in your software images, you must limit the ability of the typical network user to download and install his or her own software. If you don't, the potential for your network being exposed to viruses, malicious scripts, Trojans, and other security threats is increased.

Benefits From Enforcing Standard Software Images

Enforcing standard software images promotes:

  • System stability

  • Lower support costs

  • Reduced downtime

  • A more effective work environment

An effective way to standardize the software installed on client computers is to make it a clearly stated company policy that only standard (approved) applications will receive support.

Assembling Your Test Computers

This section discusses the need for testing your deployment strategy and outlines a method for doing this. Requirements for setting up a test lab are also presented.

Test Lab Strategies

One vital step in any deployment project is to test the deployment strategy. The most common way to accomplish this testing is to assemble a test laboratory using equipment that represents the hardware most common in your environment. It can be difficult selling this idea to management, but the payoff is well worth the investment. A thorough test program can save many hours of troubleshooting during the deployment, and can easily be credited with the overall success of a deployment project.

Most Desirable Method

Test labs can be viewed from many different aspects during planning. The most desirable method includes the following steps:

  • Gather machines that accurately represent every possible hardware configuration found in the enterprise.

  • Provide enough servers to reproduce the network structure on a small scale.

  • Provide workstations that represent every configuration found in the network.

  • Provide two workstations of each type—where possible. (This makes it possible for a developer to work on finding a resolution to an issue on one workstation, while testing continues on another.)

Often the budget allocated for testing will be very tight. If so, you will need to be creative by using each computer in the lab for a variety of roles; this will minimize the total number of computers required to reproduce your design.

Ongoing Testing

It is essential to test software and hardware changes in your environment prior to actually deploying those changes. The ability to properly test a new software application enables you to identify any issues with your software configuration, and enables you to address those issues.

Changes Requiring Testing

Software testing in your network environment may be initiated by various types of changes. Of course, testing would be an obvious step when deploying an operating system or large group of applications, but even software updates require testing. Service packs or software patches may have unpredictable interactions with existing applications. Virus updates may need to be thoroughly tested prior to deployment. Having a test lab established provides an opportunity to execute these types of testing with minimal expense and effort.

Requirements for a Test Lab

To test your applications on a new operating system you should set up a test lab. For a very basic test lab you need to acquire one computer of each type that will be supported in your network. More efficient and complex testing procedures will require more equipment, and will depend largely upon your desired schedule.

The following list briefly outlines the requirements for a test lab:

  • Space required. The physical space required by a test lab varies greatly depending on the type of deployment you are planning and the future plans for the test lab. For companies that intend to keep the lab for continual testing of software updates to the production environment, the lab will be larger than one that will only be used temporarily.

  • Personnel. The number and job functions of the people required for a test lab depends on the schedule and budget of the deployment project. Wherever possible provide dedicated staff to perform the software and hardware testing. Developers will be required to make adjustments in line-of-business applications used within the organization and to provide customization. A test lab manager is useful to provide authoritative scheduling for the resources.

  • Computers required. The computer systems used in the test lab should represent the systems used in your environment. One computer per seat is required for efficient testing. These computers must at the very least meet the minimum system requirements to run Windows XP. You should also provide a test computer for each configuration to run the current operating system and software to compare issues found on Windows XP to the current configuration.

  • Power supply. The electrical supply for the test lab must be adequate to efficiently handle the load of all hardware running simultaneously. Outlets should placed so that computers can be plugged in without moving furniture or crawling under desks if possible. Cords should never have to be deployed across walk areas.

  • Ventilation. Running multiple computers in an enclosed area will generate a great deal of heat. Air conditioning should be available to compensate for this.

  • Security. Test labs contain large quantities of expensive hardware and software. Restrict physical access to this room to prevent theft or vandalism.

Testing Your Applications

This section discusses several different methods for testing software, and outlines the responsibilities of test engineers. Planning your test framework, with a focus on earning the Designed for Windows XP logo certification, is also covered.

Testing New Applications—Exploratory Method

Microsoft uses the exploratory method for testing new applications. This method assumes that the tests will be determined as new features are discovered within the application. At the very least, you must determine whether the new application will run correctly on the chosen operating system, and that all of the functions you will need are supported. A more thorough approach would also verify that the application works correctly in all aspects of its operation on every hardware/software configuration present in your environment.

Testing Older Applications

When testing older applications that are already in use within your environment, the exploratory method will not be of much use. The exploratory method is used for new applications whose features are not clearly understood.

In the case of existing applications you are likely to already have a lot of experience using the application within your organization. What must be tested with these applications is their compatibility with Windows XP.

To accomplish testing with known applications

  • Identify the areas of primary interaction with the operating system. (For example: printing, network, file input and output, help, and so on.)

  • Once these areas are fully identified, design tests that will determine whether or not the applications are compatible with Windows XP.

Responsibilities of Test Engineers

During the test portion of your project, the test engineers will be responsible for the following:

  • Planning the test framework. The test staff must decide on a standard approach for testing applications on Windows XP. This includes deciding what fundamental tests will be used to determine application compatibility with Windows XP.

  • Establishing the test environment. Basic test software must be installed and configured on the test computers. This may include configuring the servers to replicate the organization's network structure, and configuring workstation computers with the various software images to be tested.

  • Executing the tests. Testing will normally begin with establishing the core functionality of each application and then proceed into more detail as the testing continues.

  • Reporting results. Testing results must be tracked. It is useful to establish a database for this purpose that can be accessed by test engineers, developers, and project management staff.

Planning Your Test Framework Using Microsoft Documentation

The Windows Application Compatibility team at Microsoft provides assistance and guidance for ISVs who are developing applications that will run on Windows. Toward this end, they have developed documentation that explains testing procedures and provides suggestions for how you might qualify for the Designed for Windows XP logo. Two key documents are: Designed for Windows XP Application Test Framework, and the Windows Applications Exploratory Test Procedure.

The Designed for Windows XP Application Test Framework

If you are new to software testing and want some detailed instruction on how to test applications for Windows XP, Designed for Windows XP Application Test Framework will provide what you are looking for. This document describes in detail the process to use when testing an application for the Designed for Windows XP logo and provides insight into application testing in general. It can be a very valuable addition to your deployment plans.

Note: Designed for Windows XP Application Test Framework should not be used as the only guide for testing an application. It is intended to be an addition to your overall test procedure, or used to provide the foundation for a testing procedure. You may decide to include other types of tests that apply directly to specific features of the product or your environment.

The Designed for Windows XP Application Test Framework includes the following sections:

  • Test Framework Overview. This section includes a list of the current Designed for Windows XP logo requirements, an explanation of the exploratory test procedure, and a discussion of the prerequisites for using the framework document.

  • Windows XP Testing Environment. This section details how to establish a test lab that will provide all of the support necessary for testing applications. This is particularly useful to those who are planning to implement a test lab for use with Windows XP.

  • Test Cases. This section uses a series of test cases to describe in detail how to perform tests for each requirement in the Designed for Windows XP Application Specification. (These tests are suggestions for procedures you might add to your own testing; they are not required for the logo.)

Windows Applications Exploratory Test Procedure

This document was created to explain exploratory testing— a common testing method for testing new applications on Windows XP. Exploratory testing is best used when the tester is unfamiliar with the full functionality of the product, and test cases will be defined as the tester learns about the product.

Windows Applications Exploratory Test Procedure contains five sections:

  • Introduction to Exploratory Testing. This section discusses the merits and methods of exploratory testing. This description is very helpful to anyone not familiar with this testing methodology.

  • Working with Functions. This section delves deeper into the exploratory method and how it can be used to define the functions of an application to be tested.

  • Testing Functionality and Stability. This section discusses testing how the functions of the application interact with the overall stability of the operating system.

  • Reading and Using this Procedure. This section completes the introduction to the exploratory test method. It's intended to help new testers familiarize themselves quickly with applying the exploratory method to applications on Windows XP.

  • Test Procedure. This section contains the actual test method for testing your application for compatibility with Windows XP.

This document will provide valuable insight into software testing for new applications on Windows XP. These tests should be considered as additional to any testing your organization already uses.

The Designed for Windows XP logo tells potential customers that an application has been thoroughly tested for compatibility with Windows XP. The logo displayed on the product package indicates that the software meets or exceeds every requirement in the Designed for Windows Application Specification. This means that IT professionals planning the deployment of Windows XP will have to do less testing if they select only logo applications.

Levels of Logo Certification

There are different levels of logo certification for Windows XP as shown in Table 1 below.

  • For ISVs creating new applications for the retail market, the Designed for Windows XP or Optimized for Windows XP certification levels are the best choices. For internal applications it is still recommended to strive for this level of compliance even if the application will never be submitted for logo certification.

  • The Compatible with Windows XP certification is primarily intended to be an indication that an application performs its main functions without crashing, and that any drivers it installs have been tested and proven not to cause issues for the operating system or other applications.

With the current logo program, ISVs are primarily responsible for testing their own applications. Using the same guidelines, IT professionals can easily design testing for the applications to be used in a Windows XP deployment.

Table 1 Application Compatibility Levels for Windows XP

Certification Level

Requirements

Benefits

Compatible with Windows XP

Basic functionality

Drivers are tested and certified

Listed in Windows Catalog

Designed for Windows XP

Basic functionality

Drivers are tested and certified

Meets the requirements listed in the Designed for Windows XP Application Specification

Listed in Windows Catalog

Displays Logo on package

Included in Microsoft marketing programs

Meets requirement for Microsoft Certified Partner program

Optimized for Windows XP

Basic functionality

Drivers are tested and certified

Meets the requirements listed in the Designed for Windows XP Application Specification

Meets additional compatibility requirements

Exploitive support for Windows XP features

Listed in Windows Catalog

Displays Logo on package

Meets requirement for Microsoft Certified Partner program

Deep discount and priority for Microsoft online advertising

Priority for Logo promotions

Ensures greater compatibility with future Logo requirements

Using Application Verifier to Test Applications

Anyone working toward getting an application certified in the Designed for Windows XP program will appreciate this tool. The Application Verifier (AppVerifier) tool monitors an application for known compatibility issues while you "exercise" the program. You tell AppVerifier which application to monitor, and what tests to perform while the application runs, and then you try to use each function of the application being tested. AppVerifier will report back with any problems that it finds during the process.

Two Specific Test Categories

This tool focuses on two specific test categories:

  • Tests that are difficult to perform due to vendor system limitations

  • Subtle problems that are difficult to diagnose on Windows XP.

The main window of AppVerifier is shown in Figure 1 below.

Figure 1: Using the Application Verifier tool

Figure 1: Using the Application Verifier tool

Using AppVerifier to Test for Designed for Windows XP Logo Requirements

AppVerifier can assist with testing for several Designed for Windows XP requirements:

  • 1.1 Perform primary functionality and maintain stability. Use the following tests with a debugger attached during a normal test pass: Detect Heap Corruptions, Locks Usage Checking, Detect Invalid Handle usage. These tests check for subtle application errors at the kernel level where leaks and memory corruption can occur.

Note: AppVerifier can also assist you in locating potential application compatibility issues with your existing programs when deploying Windows XP.

Identifying Resolutions to Compatibility Issues

If you are an IT professional responsible for deploying Windows XP in a network environment, application testing is only part of the story. Once you have determined that there are compatibility issues with one or more of the applications in your environment, the task becomes how to find a resolution for those issues.

This section discusses three options from Microsoft for resolving application compatibility issues. They are:

  • Using QFixApp to find compatibility fixes

  • Using the Compatibility Administration tool to distribute compatibility layers

  • Obtaining resolutions through Windows Update

Using QFixApp to Find Compatibility Fixes

QFixApp is a small application that provides an interface to the database of compatibility fixes provided with Windows XP. This tool is fairly simple to use:

  • Browse for the executable file to be fixed

  • Select the compatibility modes or fixes you want to apply.

This is a manual process, rather than the user-friendly approach provided by the Program Compatibility Wizard, but it enables you to gain a very fine level of control over the compatibility fixes applied to your application. Another benefit to QFixApp is that it doesn't depend on the application being included in the list of known applications with compatibility issues. QFixApp may be very useful when trying to determine the correct fixes to apply to custom in-house applications. The main interface of QFixApp is shown in Figure 2 below.

Figure 2: Using QFixApp

Figure 2: Using QFixApp

How to Apply Fixes with QFixApp

  • Begin by deciding whether you want to use a compatibility mode or a compatibility fix. The compatibility modes are collections of fixes that can be applied all at once. There is a tab for Modes, and one for Fixes.

  • It is usually best to begin with a compatibility mode if you know that the application ran on another version of Windows. Using the Fixes tab you can select individual fixes that comprise the compatibility modes. This is especially useful when your application only requires a few compatibility fixes to run.

  • When you have made your selections and are ready to try them out, click the Run button, and QFixApp will test the combination of fixes.

Using CompatAdmin to Distribute Compatibility Layers

In a larger network environment there may be many applications that require the help of Windows XP application compatibility technologies. And because of the distributed nature of large networks, getting the correct bundle of compatibility fixes to users may be difficult and time consuming.

Microsoft has provided the Compatibility Administration Tool (CompatAdmin) to help administrators easily assemble and distribute packages of compatibility fixes. The main interface of CompatAdmin is shown in Figure 3 below.

Figure 3: Use CompatAdmin to assign bundles of compatibility fixes to applications.

Figure 3: Use CompatAdmin to assign bundles of compatibility fixes to applications.

Over 165 Fixes Are Available

CompatAdmin provides an interface for browsing and editing the Windows XP compatibility fix database. The tool allows you to select any executable file and apply one or more of the over 165 fixes available in the operating system. Other compatibility fixes will be added through the Windows Update feature as they are created. You can use QFixApp to identify which combinations of fixes allow your application to work on Windows XP. When you’ve applied all of the fixes needed to run the application, you can use CompatAdmin to create a package to deploy these fixes to other Windows XP machines.

Obtaining Resolutions Using Windows Update

Windows Update is an exciting application compatibility feature in Windows XP. Using Windows Update product updates can be automatically—and transparently— downloaded and installed. You have the option of having Windows Update automatically install the updates, or to simply download them and be prompted whether or not to install at that time.

Windows Update and Dynamic Setup

Windows Update also plays a part in the installation of Windows XP. If you choose to use Dynamic Setup and have a working Internet connection, Windows XP Setup will access the Windows Update Web site to check for any updated files that would benefit your computer. These updates can be anything from security updates to product fixes from ISVs. This feature will help to ensure that the version of Windows XP you are installing has the latest files and updates available right out of the box.

A Single Distribution Source

As part of the new Windows Update, Microsoft will offer ISVs and IHVs the opportunity to distribute their software updates through this channel. This is intended to provide a single-stop location for software updates.

Summary

When deploying Microsoft Windows XP to a network environment, testing is crucial to your success. Microsoft has provided numerous resources to assist with the testing and deployment of Windows XP. AppVerifier provides a useful augmentation to your standard testing procedures to locate common compatibility issues, or subtle, hard-to-locate kernel issues. QFixApp can be used to determine the combination of compatibility layers or compatibility fixes that will resolve an application compatibility issue. CompatAdmin can be used to distribute the package of compatibility fixes to other Windows XP computers once they have been identified.

Together, these resources can make it easier to test the software applications in use in your network environment prior to the deployment of Windows XP.

See the following resources for further information: