Update Management Process

Updated : June 1, 2007

On This Page

In This Module
Objectives
Applies To
How To Use This Module
Overview
Determine the Appropriate Response
Plan the Release
Build the Release
Acceptance Testing
Summary
Handover to Deploy Phase

In This Module

This module describes the third phase—Evaluate and Plan—of the four-phase update management process. The Evaluate and Plan phase is concerned with deciding whether to deploy an update, determining what is needed to deploy it, and testing the software update in a production-like environment to confirm that it does not compromise business-critical systems and applications.

The purpose of this module is to describe the principles of the Evaluate and Plan phase of the update management process, and to introduce the types of tasks that enable evaluation and planning to be carried out using Microsoft Windows Server Update Service (WSUS) and Microsoft Systems Management Server (SMS).

Note: The next release of SMS, entitled System Center Configuration Manager 2007, is now available for download at https://www.microsoft.com/technet/sms/2007/evaluate/download.mspx. With major investments in simplicity, configuration, deployment and security, Configuration Manager 2007 dramatically simplifies system deployment, task automation, compliance management, and policy based security management allowing for increased business agility.

Without an evaluation and planning process, you will not have a clear set of criteria to help decide whether to deploy an update, you will not know what is needed to deploy an update, and will not have procedures in place to build reliable, well-tested release software.

Objectives

Use this module to:

• Determine whether an update deployment is actually required.

• Plan the release of the software update.

• Build the release.

• Conduct acceptance testing of the release.

Applies To

This module applies to all Microsoft products and technologies.

How To Use This Module

While the basic tasks needed to carry out evaluation and planning using WSUS and SMS are described in this module, more detailed instructions can be found in the Technical Libraries listed below.

To gain the most from this module, you should:

• Read the Introduction portion of the "Update Management Process" module. This will give you an overview of all phases in the four-phase update management process and provides an introduction to the tools available to support update management in Microsoft Windows operating system environments.

• Use the WSUS and SMS Technical Libraries for more detailed reference materials. These libraries are located at:

  • Windows Server Update Services (WSUS) Technical Library

  • Technical Library for Systems Management Server 2003

Overview

The Evaluate and Plan phase is the third phase in the update management process, which is shown in Figure 1.

Figure 1 The update management process

In this third phase, you evaluate the software update and—assuming that it is approved for deployment—plan for its deployment into the production environment.

The entry point for the Evaluate and Plan phase is a request for change (RFC) for a software update that has been identified as relevant to your production environment.

By the end of the Evaluate and Plan phase, you should have determined whether the change request should be classified as an emergency, reviewed and approved the request, and determined the tasks necessary to deploy the approved changes into production. You should also have tested the software update in a production-like environment to confirm that it does not compromise business-critical systems and applications.

This module focuses on the key requirements for evaluation and planning, which are to:

• Determine the appropriate response.

• Plan the release of the software update.

• Build the release.

• Conduct acceptance testing of the release.

Determine the Appropriate Response

The RFC determines the sort of change required in the production environment—such as deploying a software update, applying countermeasures that mitigate a vulnerability, or both—and describes the required change, so others can act on it.

The first step in evaluating and planning is to review the RFC and determine the most appropriate response to a software vulnerability or threat. This will involve:

• Prioritizing and categorizing the request.

• Obtaining authorization to deploy the software update.

Prioritizing and Categorizing a Request for a Software Update

Before a request for a software update can be authorized, its priority and category need to be determined. Although priority and category are initially assigned by the change initiator and included in the RFC, those assignments have to be reviewed and either agreed to or changed before the change request can be authorized.

Prioritizing a Software Update

The level of priority is particularly important because it determines how quickly a software update passes through the change process. The following considerations can help you to determine the priority level of a software update:

• What are the critical business assets? Will these be exposed to a potential security breach or system instability until the software update is installed? You should prioritize change requests based on the impact of updating or not updating high-value assets.

• Will the software update apply to a system running a business-critical service, such as a line-of-business (LOB) application, which has been the target of attackers in the past? This can be a good reason to raise the priority of a change request.

• Have you deployed countermeasures that should minimize the exposure of a particular security vulnerability? This can lower the priority of the change request, although it may still be appropriate to deploy the software update to eliminate the vulnerability.

• What is the threat of the vulnerability in question to the production environment? Many security bulletins and related software updates might apply to only a few computers in your environment. If the threat of the vulnerability is low, that might lower the priority of the request.

The following tables can help you to assess the priority of the request in relation to other requests. Table 1 maps priority levels to recommended time frames and minimum recommended time frames.

Table 1: Update Priorities and Recommended Deployment Time Frames

Priority	Recommended Time Frame	Minimum Recommended Time Frame
Emergency	Within 24 hours	Within 2 weeks
High	Within 1 month	Within 2 months
Medium	Depending on availability, deploy a new service pack or update rollup that includes a fix for this vulnerability within 4 months.	Deploy the software update within 6 months
Low	Depending on availability, deploy a new service pack or update rollup that includes a fix for this vulnerability within 1 year.	Deploy the software update within 1 year, or you may choose not to deploy at all.

Table 2 gives examples of factors that might raise or lower the priority.

Table 2: Factors Affecting Update Priorities

Environmental/Organizational Factor	Priority Adjustment
High-value or high-exposure assets impacted.	Raise
Assets historically targeted by attackers.	Raise
Mitigating factors in place, such as countermeasures that minimize the threat.	Lower
Low-value or low-exposure assets impacted.	Lower

Emergency Change Request

If the vulnerability that the security update addresses is already being exploited or is about to be exploited, or if the update corrects a system instability being encountered in the production environment, then you might need to classify the request as an "emergency," giving this request priority over all other changes happening within the production environment.

Categorizing a Software Update

The category of a software update is important, because it helps those reviewing the change to understand the impact it will have on systems and services within the production environment. To establish the category (or impact) of the change request, you will need to determine:

• On which machines the software update needs to be installed and the roles (business criticality) of those machines.

  A software update that requires a business-critical computer to be rebooted, for example, will have a greater impact than one that does not.

• Whether additional changes will be needed to support the deployment of the software update.

  If, for example, the software update applies only to the current service pack, and that service pack is not installed on certain production systems, it may not be possible to protect those systems against a particular security vulnerability. In this case, the impact and hence the category of the change request would be greater, because both the service pack and the software update would need to be deployed.

• Whether the software update can be uninstalled once it has been installed.

  If not, then it presents a greater risk to the production environment than one that can be successfully removed. Although it may be necessary to deploy such software updates to provide protection from a particular security vulnerability or to address a particular system instability, the category of the request needs to reflect this.

• The likely impact on the network infrastructure.

  Deploying a large software update to many computers simultaneously could degrade network performance and adversely affect the proper operation of your entire environment. You should closely review all software update documentation, and always be aware of the software update's size and the number of computers that will receive it. This information can assist with properly scheduling the release.

• Whether certain services need to be stopped, paused, or closed during installation.

  This may affect an organization's critical services or prevent an end user from working on the computer while the installation is taking place.

More information about setting the priority and category of a change request can be found in the Change Management service management function (SMF) at https://www.microsoft.com/technet/itsolutions/cits/mo/smf/default.mspx.

Obtaining Authorization to Deploy the Software Update

Once the change request has been prioritized and categorized, it needs to be reviewed and authorized, before the software update can be deployed into production. To get the change request authorized, you need to:

• Determine who should be involved in the decision-making process.

• Review the change request, assess the risks and consequences of deploying the software update, and select the most appropriate course of action.

• Identify who will be responsible for getting the software update deployed to all impacted systems.

Determining Who Needs to Be Involved

It is important to determine who should be involved in reviewing and authorizing a software update for deployment into production. For non-emergency updates, review and authorization should be done by a change advisory board (CAB), made up of representatives from all areas of the business that will be affected.

CAB members should include individuals who have experience in the specific technologies and services that will be used to deploy the update. Representatives from the business, network, security, service desk, and technical support teams should also be included in this group.

More information about CABs and how they can be formed can be found in the Change Management SMF, which can be downloaded from https://www.microsoft.com/technet/itsolutions/cits/mo/smf/default.mspx.

Emergency Change Request

When a quick decision needs to be made about a critical request—one designed to close a security vulnerability or avoid a critical system failure—authorization should be done by the CAB emergency committee (CAB/EC).

A CAB/EC should be composed of people who possess the authority to approve emergency changes and who will be available to make quick decisions. More information about CAB/ECs can be found in the Change Management SMF, which can be downloaded from: https://www.microsoft.com/technet/itsolutions/cits/mo/smf/default.mspx.

Review Change Request

Once you have assembled the appropriate people, you need to assess the risks and impact of the software update to the production environment and determine whether it should be deployed. In making this decision, you will need to discuss the following issues:

• What else is happening in the production environment?

• What is the impact of applying or not applying the software update?

• What is the anticipated cost of deploying or not deploying the software update?

• What are the steps that can be taken—if any—to mitigate the exposure to a security vulnerability or system instability, while the software update is being deployed?

• What is the impact of computer downtime? It will be necessary to compare and consider the risks of postponing the deployment of a software update versus the risks incurred by causing computer downtime when deploying a software update to your environment.

• What will be the best and most effective mechanism for deploying the software update?

• Are there any known issues or side effects with the software update, and does it necessitate a system restart?

• Are enough resources available to deploy the software update or deal with any issues experienced during deployment?

• How will you address any dependencies or prerequisites that need to be met before the software update can be deployed?

Although the best response to a software vulnerability is to deploy the software update that resolves the issue, sometimes it may be preferable to deploy a short-term countermeasure—such as closing network ports or shutting down external access to systems—while the software update is being rolled out to systems within the production environment. Applying countermeasures may have several benefits:

• The majority of software updates require that target computers be restarted before the installation is complete and the computer is safeguarded. If you are prevented from immediately deploying a software update to your environment, because computer restarts are limited to specific maintenance windows, then implementing recommended countermeasures can effectively safeguard your computers until the software update can be deployed. Alternatively, sometimes security updates can be deployed and the automatic restart suppressed. In this instance, the security update could be installed during normal hours and then the computer restarted at a time suited to the maintenance window.

• Countermeasures may have a lower risk, and can be applied more quickly and with less testing than the software update itself. It may be significantly easier, for example, to disable network ports, or to shut down services or systems that are exposed to a particular security vulnerability, and apply the software update later.

Implementing computer hardening countermeasures can often protect computers from many common security vulnerabilities. Blocking certain network ports and disabling unused services are just two of the countermeasures that, when implemented, can effectively safeguard your computers.

For more information on computer hardening countermeasures, see the "Threats and Countermeasures" guide at https://www.microsoft.com/downloads/details.aspx?displaylang=en&familyid=1b6acf93-147a-4481-9346-f93a4081eea8

Note: Even if countermeasures are deployed to reduce the exposure to a security vulnerability, the security update should still be scheduled for deployment.

For example, if systems remain in a non-updated state and a computer that is infected with a worm or virus is introduced into the network, the infection will quickly spread to all unprotected systems. The deployment of countermeasures should simply lower the priority of the change request, rather than remove the requirement for the software update.

Decide Who Will Own Deployment of the Software Update

Once an agreement has been reached to deploy the software update and to use any countermeasures (if appropriate), you need to identify who will take responsibility for making sure that these changes happen. This person will need to:

• Develop a plan for making the required changes.

• Determine and obtain the resources required.

• Arrange for the development of any necessary scripts, tools, and documentation that will be necessary to deploy the changes.

• Ensure that adequate testing is carried out.

• Ensure that the changes are deployed into production.

• Assess the success or failure of deployment.

Without a designated owner to oversee the above activities, there is a risk the software update might not be deployed. The designated owner is typically the Release Manager role, referred to in the Release Management SMF at: https://www.microsoft.com/technet/itsolutions/cits/mo/smf/default.mspx.

Plan the Release

Release planning is the process of working out how to release the software update into the production environment.

There are essentially three stages involved when planning the release of a new software update:

• Determining what needs to be updated.

• Identifying the key issues and constraints.

• Building the release plan.

Determining What Will Be Updated

Deciding what needs to be updated requires an accurate and up-to-date knowledge of what is present in the environment.

In a WSUS environment, administrators may need to re-scan using Microsoft Baseline Security Analyzer (MBSA) to establish those systems that require the software update. If SMS is deployed, information retrieved by the inventory tools and client agents can be used to determine the machines on which the update needs to be installed.

For more information about these tools, see Windows Server Update Services (WSUS) Technical Library and Technical Library for Systems Management Server 2003.

Identifying the Key Issues and Constraints

There may be a number of issues and constraints that dictate the steps necessary to fully deploy the software update into production. For example, when tasked with deploying the software update, you need to consider the following:

• How much time should you give users before the software update is installed automatically? The amount of time permitted will depend on a number of factors, including user roles and responsibilities, and the nature of the system instability or security vulnerability that the software update is designed to address.

• Figure 2 shows an example software update deployment timeline (a service level agreement [SLA]), which indicates that servers must be updated within seven days of the arrival of a software update that is not deemed business critical. Administrators are granted permission to start updating within the seven-day window, but need to be aware that computers will be force-updated or disconnected from the network after the time period expires.

  

  Figure 2 An example update deployment SLA for servers

A similar compliance timeline may be applied to workstations in your environment, although the specific response times and actions will vary.

• Certain software updates require administrative rights on the computer on which they are being installed. Most end users will not have local administrator rights, so the tool used to install the software update will need to be able to acquire elevated rights and privileges to install the software update on client computers.

• If the software update requires a certain amount of disk space to install, or the software update is to be cached locally prior to installation, then a check needs to be made on the amount of free disk space on each client computer.

• If the software update is large—for example, several megabytes in size—mobile clients may take some time to download it. If the software update is not classified as an emergency, it may be appropriate to postpone installation on those clients until they are physically connected to the network.

• Business-critical computers may have specific times at which changes and computer restarts are permitted (outage windows). The deployment of a software update and any system restarts that are required as a result will need to be scheduled within these outage windows.

• If client computers are locked down through the use of certain Group Policy settings, this may affect the software update's ability to install correctly.

• If the product to be updated was deployed using Windows Installer, the Windows Installer may require access to original installation files. These files need to be in the same location as they were when the product was originally installed, if you are performing an unattended installation of the software update. If the product was originally installed from physical media—a CD drive, for example—Windows Installer will try to find the original files on the currently inserted CD.

• If an administrative image was used for the original installation of a Microsoft Office product, for example, you will need to perform an administrative installation of the software update to the image rather than apply it directly to the client. More information on the issues around updating an administrative image is available at https://www.microsoft.com/technet/sms/20/smsoffxp.mspx.

• If there are applications that were installed on a per-user basis, rather than a per-computer basis for all users, you should reinstall the application on a per-computer basis and then apply the software update to the new installation.

If you use SUS, and wish to avoid deployment outside the outage windows, you will need to use Group Policy objects (GPOs) to specify that updates should be downloaded, but not installed. Once you know an update has been downloaded, you should log on to each business-critical server and start the installation within the permitted outage window. For workstations, there is also a GPO setting to ensure that computers that have missed a scheduled installation (for example, because the client computer was turned off) will automatically reschedule the installation when the Automatic Updates service starts on boot up. For more detailed information about using WSUS in the Evaluate and Plan, see Windows Server Update Services (WSUS) Technical Library and for information on how to use SMS to schedule deployment, see Technical Library for Systems Management Server 2003.

Emergency Change Request

If you have a change request that indicates the software update is an emergency, then you consider the following factors:

• You may need to deploy the software update in a significantly shorter time span than normal. Full deployment, for example, may be expected within 24 hours of the approval of the change request.

  Figure 3 shows an SLA involving a software update deployment timeline. The timeline indicates that all servers must be updated within eight hours of the arrival of a critical software update notification.

  

  Figure 3 Example of an eight-hour emergency deployment SLA

  In the preceding example, once the organization has been made aware of the software update, updating of business-critical servers must begin within two hours (12:00 as shown) with 98 percent of all servers brought into compliance by 18:00. The remaining servers need to be brought into compliance by 22:00 or risk being disconnected from the network. A similar compliance timeline may be enforced for workstations in your environment, although the time window and speed of response may be different for your organization.

• The outage windows that govern when certain business-critical systems can be updated or restarted will need to be overridden to allow the software update to be deployed within the agreed-to time frame.

• You will need to apply the software update to all affected systems, regardless of their connection to the network.

If you use WSUS you can use Group Policies to force clients to install a software update before a regularly scheduled maintenance window. Before doing this, you will also need to make sure that you force a replication on any child WSUS servers, which are normally scheduled to synchronize updates at network quiet times, using the Synchronize Now option in the WSUS Server Administration Page. For more detailed information about using WSUS in the Evaluate and Plan phase, see Windows Server Update Services (WSUS) Technical Library.

In an SMS environment, you should configure the SMS intersite senders to allow high-priority package/advertisement transactions to replicate at all hours of the day. It is important that the high-priority setting is only used for software update packages and advertisements. For more detailed information about using SMS in the Evaluate and Plan phase, see Technical Library for Systems Management Server 2003.

Building a Release Plan

This is the point at which you will need to plan and determine the order in which the computers within your production environment will deploy the software update. The following are some of the issues you may need to consider when building the release plan:

• If the software update applies to all servers within the production environment, should those running WSUS and SMS or monitoring tools be updated first?

  Updating the management infrastructure first ensures that administrators can use these services to monitor the progress of the deployment.

• Are there any business reasons why one part of the production environment should be updated before another?

  There may be compelling reasons to apply the software update to computers that are at risk from a security vulnerability or potential system instability and then, once these computers are updated, continue the rollout elsewhere.

• What impact does the available network bandwidth between sites have on the rollout order?

  It may be difficult to get the software update out to some sites as quickly as others, because of network bandwidth constraints. The software update can be deployed more quickly to sites with good network connectivity than those where network availability is limited.

Finally, you will need to determine how and when information about the software update, its severity, impact, and the steps that need to be taken to deploy it, will be communicated to users, the business, and the service desk.

Emergency Change Request

In the case where the change request is an emergency, then you should consider the following:

• If management architecture components such as Microsoft Operations Manager (MOM), WSUS and SMS 2003 site servers also need to be updated, it may be appropriate to plan to have these computers updated manually—by local administrators—to ensure that these servers are not restarted during the rollout of the software update to other computers within the production environment.

• If one site or group of computers has already suffered from a security breach or system instability that the software update addresses, deployment of the software update should be directed to those computers after appropriate remediation steps has been taken (Example: Virus removal, and etc..)

Build the Release

With a release plan in place, the next stage of the process is to develop the scripts, tools, and procedures, which the administrators will use to deploy the software update into the production environment. The tasks and activities that need to be carried out here will primarily depend on whether the software update can be deployed using WSUS or SMS 2003.

If you use WSUS, note that updates are downloaded directly from the public Microsoft Update servers and are already packaged in an executable (.exe) format, so you don't need to do any additional work to repackage them for deployment. To find more detailed information on using WSUS in the Evaluate and Plan phase, see Windows Server Update Services (WSUS) Technical Library.

If you use SMS, building the release is simplest if the update can be deployed using the SMS 2003 Distribute Software Updates Wizard. This wizard can automatically create the SMS package, as well as the program required to distribute and install the software update. All security updates that apply to a specific product and service pack combination can then be combined into a security rollup package, which simplifies management and administration, as well as significantly reducing the number of computer restarts required. For updates that cannot be deployed using the SMS 2003 Distribute Software Updates Wizard, you need to use standard SMS software distribution procedures to create a package and advertisement, specify the batch file, Microsoft Installer (MSI) file, or an executable that will be run on the client computer, and if the update is not supplied with a setup executable, you will also need to build an executable using the MSI authoring tools. To find more detailed information on using SMS in the Evaluate and Plan phase, see Technical Library for Systems Management Server 2003.

Acceptance Testing

Up to this point, the purpose of testing has been to confirm that the software update and the release package work correctly within a development environment.

Acceptance testing allows developers and business representatives to check that updates work in an environment that closely mirrors production and that business-critical systems will continue to run successfully once the software update has been deployed. Administrators, together with business representatives, should draw up a set of tests that will be performed when a software update is regarded as business critical, and a more detailed set of tests that can be used when the software update has a lower priority.

However critical the software update is, however, you should always carry out a minimum level of testing to show that:

• Once installation is complete, the computer will reboot as it is designed to.

• The software update, if it is targeted at computers connected across slow or unreliable network connections, can be downloaded across these links and, once this completes, it successfully installs.

• The software update is supplied with an Uninstall routine, which can be used to successfully remove the software update.

• Business-critical systems and services continue to run once the software update has been installed.

Before you deploy the software update into production, it is important to collect information about any troubleshooting steps, procedures, and tools that are used during testing, and to make these available to service desk support staff and the operations team.

You should expect that testing will result in the creation of:

• Internal Knowledge Base articles, which describe standard troubleshooting steps, together with any associated workarounds.

• A list of contacts and an escalation path.

• Scripts, rules and information (such as counters, events, and thresholds), which will enable your operations staff to effectively monitor the release in production.

No matter how much testing is performed, rolling out a software update into production often produces effects that can never be anticipated or replicated in a lab environment. To avoid impacting a large number of client computers with a potential failure, administrators should consider rolling out the software update to a small representative sample of computers and confirming that business-critical systems and applications continue to run, before deploying it to the entire organization.

Summary

The following are the key things to remember from the Evaluate and Plan phase:

• You need a formal process to determine whether it is in the best interests of the business to deploy the software update.

• You need to have an identified owner of the software update who will be responsible for ensuring that it is deployed.

• When you have an approved software update, you need to plan how to get it into production.

• You need to test the package in a lab and, if needed, pilot test it in a production environment to confirm that it does not compromise LOB applications.

Handover to Deploy Phase

The trigger for handover to the Deploy phase of the update management process is that:

• The package is ready for deployment into production.

• You have received approval to deploy the software update into the production environment. For more information about the Deploy phase, read the module, "Update Management Phase 4 - Deploy."