Security WatchI Just Got a Security Bulletin. Now What?
Christopher Budd
The Microsoft monthly security bulletin release process, established in October 2003, was a response to customer requests. One of the chief requests was for predictability in the release schedule of security bulletins. We've been able to use that regularity and predictability to hone and improve our processes.
The Microsoft Security Bulletin site is where you can find the most recent security bulletin and search past bulletins, as shown in Figure 1. An example bulletin, in this case for August 2006, is shown in Figure 2.
Figure 1** Microsoft Security Bulletin Site **(Click the image for a larger view)
For many customers, the Microsoft monthly security bulletin has helped foster more mature processes for deploying Microsoft security updates. With security bulletins being released on a predictable day, customers have used this opportunity to build their own regular processes for handling them. These customers have created timelines and steps in their own security update evaluation and deployment processes that integrate the predictable timelines around the Microsoft monthly security bulletin release. Customers use that process to analyze, test, and deploy updates in a systematic fashion that helps to increase their overall security posture and decrease downtime and disruption.
Figure 2** August 2006 Security Bulletin **(Click the image for a larger view)
However, some customers proceed through security update evaluation and deployment without a set procedure. While it is possible to deploy updates in an unsystematic fashion and not experience any problems, it fundamentally runs counter to what we consider to be best practices for managing security updates.
We've provided useful information about how and when we release bulletins, but we have not provided much guidance on what you should be doing with the bulletins and how you can integrate them into your own processes. Some customers may not be aware of the processes that may benefit them, or the best way to build them.
Therefore, in this column I will outline the steps and processes that we recommend for your evaluation and deployment of security updates. Organizations of any size can follow these steps. For some larger organizations, it will be appropriate to divide the steps among different groups, based on the internal structure. For smaller organizations, all these steps can be carried out by one group or even by one person. Regardless of the allocation of responsibilities, all these steps should be explicitly accounted for in your evaluation and deployment processes. Figure 3 provides an overview of the process.
Figure 3** Security Bulletin Assessment and Deployment Process **
1. Receive Advanced Notification
The first step in your monthly security bulletin process should be to ensure that you receive the Advanced Notification of coming security updates. Microsoft posts information on its Web site at approximately 10 A.M. Pacific Time (GST -8) three business days before the standard monthly release, which is the second Tuesday of each month; therefore, the Advanced Notification information is posted on the prior Thursday.
The goal of the Advanced Notification is to provide information that will help deployment planning ahead of the actual release. The level of detail included is balanced against a need to protect customers until the release of the security updates by not disclosing any information that could facilitate attacks. The Advanced Notification aggregates information about the upcoming updates by product name without information about specific versions (for example, Microsoft® Windows® or Microsoft Office). Each of these entries details how many bulletins for that product will be released, the highest maximum severity of those bulletins, whether any of the updates will require a restart, and which of our detection tools will apply to those bulletins.
To ensure that there are no surprises and to minimize possible confusion, the Advanced Notification also provides information about other updates not associated with security bulletins that will be released on the same day. Specifically, it details how many High Priority Non-Security Updates will be released through Microsoft Update (MU) and Windows Update (WU), as well as any updates to the Microsoft Windows Malicious Software Removal Tool.
Finally, we will provide information about any issues that might cause delays in your deployment, as we did for the "Send as" permission change in MS06-019.
Each month the new Advanced Notification is posted, at which time a notification is sent out through the Security Notification Service Comprehensive Edition (SNSCE). You can register for the SNSCE.
2. Assess the Potential Impact
Once you receive the Advanced Notification, your first step should be to assess the relevance of upcoming releases in your environment. Even though the Advanced Notification contains limited details about the products affected, it presents enough information that you should be able to gauge whether upcoming bulletins will be relevant for your environment.
For example, suppose that Advance Notification notes two Microsoft Exchange bulletins that have a highest maximum severity rating of critical and that require a restart. If you don't run that program, you'll know right away that those bulletins won't be applicable to your environment. Conversely, if you run Exchange Server 2003, you will know that on Tuesday you might have up to two bulletins rated as critical requiring restarts.
For planning purposes, think of the Advanced Notification as a tool to help you determine the maximum possible impact on your organization in terms of numbers of bulletins, their severity, and their impact at deployment regarding restarts. Once you've assessed the relevance and maximum possible impact of the updates, you can create a schedule which should include the following action items:
- Personnel for risk assessment, testing, and deployment
- Test plans and test lab time
- Deployment plans and scheduled restarts (where appropriate)
All scheduling at this point should be tentative since information in the Advanced Notification is subject to change. Also, because of the limited nature of the information in the Advanced Notification, you may make changes once you have the full details.
Finally, if there is any additional information, such as functionality changes, you should use the time between the Advanced Notification and the actual security bulletin release to research the issue and evaluate its possible impact to your environment. You might need to revise your tentative schedule based on your research.
3. Receive New Microsoft Security Bulletins
On the second Tuesday of the month Microsoft releases its security bulletins. 10 A.M. Pacific Time is the target for release, but as the release is part of a large, complex, interconnected process, it might be delayed for a variety of reasons. In this release, security bulletins, security updates, updates to detection and deployment tools such as the Microsoft Baseline Security Analyzer (MBSA) and Windows Server® Update Services (WSUS), and new detection tools such as a new version of the Enterprise Scan Tool (EST) are all posted on Microsoft Web sites.
The Microsoft security bulletin page is enabled to support RSS feeds. When security bulletins include updates, our RSS feed is also updated which in turn will update RSS aggregators and viewers. In addition, notification is sent out using both the Security Notification Service (SNS) and the SNSCE. Notifications are sent out through both e-mail and MSN® Alerts.
To learn about the availability of new security bulletins, you should register for all of the notification mechanisms offered. You can subscribe to the RSS feed and for both SNS and SNSCE.
4. Assess the Relevance of Security Bulletins
It is important that you clearly assign the responsibility for reviewing the new security bulletins and assessing their relevance for your environment. As part of this assessment, you can take the full specific information from the security bulletins to make revisions and adjustments to your tentative scheduling as appropriate. Going back to the example, if you notified your Exchange administration team of a possible critical security update, but you then find that Exchange Server isn't affected, you can adjust your scheduling of those resources.
Once you have assessed the relevance of the security bulletins for your organization, you can then move ahead with those that are relevant in your processes.
5. Perform Risk Assessment and Rate Bulletins
At this point you need to perform a risk assessment of the relevant bulletins. In organizations with divided responsibilities, this step is often performed by the security group. While each Microsoft security bulletin contains a maximum severity rating, that rating reflects the highest severity of all vulnerabilities in all affected products. The Microsoft severity rating that applies to your organization may be different from the maximum severity rating listed, depending on the severity of the issue on the versions in your environment. It's important to look at the full Severity Ratings and Vulnerability Identifiers table under the Executive Summary section of the security bulletin and obtain the specific vulnerability information for your particular versions.
After that, you should review additional technical information such as Mitigating Factors and technical details from the FAQ for each vulnerability and make adjustments to your risk assessment based on your environment and policies. Your policies may also dictate that your assessment account for nontechnical, environmental factors such as the threat environment or the criticality of certain applications to your organization.
At the end of this process, you should have a risk rating for each of the bulletins and the updates associated with them. Continuing with our example, the Exchange administration group may rate a security update as "moderate risk" based on the changes detailed in the security bulletin. Based on your policies, you should make updates and changes to your schedule as appropriate. For example, your policies may dictate that moderate risk updates require one or two more days of testing than low risk updates.
In some cases, your policies may dictate that for bulletins with a certain risk rating you also need to consider implementing workarounds. For example, your organization's policy may state that for all security updates you rate as critical, you should review and deploy workarounds as soon as possible. In that case, you would perform an additional risk assessment on the workarounds to determine what to implement and when.
At the end of this process, you should have a list of updates to be deployed in your environment with associated risk ratings for each. In addition, you should have a list of workarounds to be implemented which should be used to update your testing and deployment schedule. The changes to your schedule should reflect the timelines specified by your policies. For example, you may have a policy that dictates that for all critical security updates, workarounds must be deployed within 24 hours and updates deployed within seven calendar days.
6. Determine the Possible Impact of Security Updates and/or Workarounds on the Organization
Understanding the impact of security updates or workarounds on your environment is a critical step in the evaluation process. In organizations with divided responsibilities, this step will often be performed by the group that manages the systems directly affected by the security updates or workarounds.
Understanding the possible impact helps you to understand and determine the risk posed by the security updates or workarounds themselves. Information can be found in the security bulletin in the "Frequently Asked Questions (FAQ) Related to This Security Update" section. Information about how the security update addresses the vulnerability is located within the Vulnerability FAQ under "Vulnerability Details."
The goal of this step is to determine a risk rating for the updates and workarounds. Continuing with the previous example, the Exchange administration group may rate a security update as "moderate risk" based on the changes detailed in the security bulletin. Based on your policies, you should revise your schedule as appropriate. For example, your policies may dictate that "moderate risk" updates require one more day of additional testing than "low risk" updates.
You may also need to revisit the decisions around deploying workarounds based on the impact of your risk rating on your schedule. For example, you may find that you need to extend testing by two days and this leaves your systems unprotected for longer than your policies allow. In that case, you would then revisit the deployment of workarounds to be in compliance with your policies.
Finally, you will use the information that comes from this step when you move on to developing a test plan for the updates.
7. Define a Test Plan
To be effective, testing must be systematic and must be built with a meaningful plan. Your test should target areas that would be most likely to exhibit problems in your environment. Since each environment is different, we can't offer a standard template for an effective test plan.
For organizations with divided responsibilities, the test plan will often be designed by more than one group. The group that manages the technologies or systems directly affected by the security updates or workarounds can provide its particular expertise. The testing group can offer expertise on building test cases, options for test tools, and an understanding of practical timelines.
During this phase you should note how the test plan impacts your risk rating of the security updates or workarounds and make changes to your testing and deployment schedule accordingly. For example, if you determine that you cannot develop a test plan to test an update as fully as you would like, you may increase its risk rating, choose to delay the deployment, and implement workarounds in the interim.
8. Develop a Deployment Plan
Deployment is the process that actually implements the protection that the security updates or workarounds provide. Because deployment is really the ultimate goal of the process, understanding the deployment methods available and factoring them into your assessments is as important as a security risk assessment. In organizations with divided responsibilities, determining possible deployment methods for security updates is often performed by the groups that are responsible for managing the software update infrastructure, such as the group that maintains Systems Management Server (SMS). Groups that have oversight of configuration management technologies such as Active Directory may be involved in determining possible deployment methods for workarounds.
In this step, your goal is to understand the possible deployment methods and to build from that a plan that you will use to deploy the security updates or workarounds. An important thing to understand in this step is how the possible deployment methods may impact your schedule and make any necessary changes. For example, if a security update is not supported by WSUS and that is your primary deployment method, you may determine that it will take you two days longer to deploy the update than originally planned. You may, in turn, decide to implement workarounds to provide necessary protections during this deployment window.
Information regarding deployment methods is contained in the security bulletin in the "FAQ Related to This Security Update" section. Information about ways to implement workarounds is listed with each workaround; individual workarounds are associated with each vulnerability.
This concludes the planning stages. At this point, you should have a schedule that reflects all elements of your assessments and planning regarding security risk, risk rating of the security updates and workarounds, testing, and deployment. As you can see throughout these stages, they are intricately interrelated and not necessarily linear. In some organizations, these stages occur simultaneously, whereas in others they are sequential. You should decide on the implementation of these steps based on the policies, needs, and resources of your organization. The most important thing for these steps isn't the specific structure and order, but rather that you ensure that these different stages are able to inform and respond to one another. The key for any implementation is to remain flexible and adaptable.
9. Test Updates and Workarounds
In the testing stage, you'll put the security updates and workarounds through the test plan defined earlier in your testing environment. The goal in your testing environment is to replicate the critical elements in your production environment. In testing the security updates or workarounds, you are striving to find any issues that might occur before you deploy to your production environment.
As problems are identified in testing, you should determine the severity of the issue and the steps needed to resolve them. You may discover minor issues that have an acceptable level of impact, whereas others will require resolution prior to deployment of the updates. If you choose to accept the issue, document it for your support staff.
To resolve the issue, you can open a case with Microsoft Product Support Services (PSS), which offers no-charge support for issues related to security updates. You can find information on how to contact PSS at support.microsoft.com/security. Because resolving issues can add time to testing and thus delay deployment, before you choose to resolve an issue you should consider the potential impact to your schedule.
When the testing is complete with all issues either resolved or documented, you can then certify the security updates or workarounds for deployment in your production environment.
10. Deploy Updates and Workarounds
After security updates or workarounds have been certified by testing, you can proceed with the deployment process. During deployment, use the plan to guide you as you actually apply security updates or configuration changes to systems. The groups that were involved in developing the plan are usually responsible for the actual deployments.
If you encounter problems in the deployment-for example, a security update fails to install properly through your update management infrastructure-you'll need to examine the problem carefully. If the problem introduces delays, you may want to re-examine options and/or consider workarounds in order to protect your systems.
Ideally, your testing process will identify all issues that might occur in production before deployment. However, if you do encounter issues after deploying into your production environment, you should follow the same steps you would if you encounter an issue in testing. Evaluate the issue and decide whether to accept or resolve it.
If security updates are successfully applied, then the protections are in place and the ultimate goal of these processes has been met. However, if they are not successfully applied, your systems are still vulnerable.
Because some systems may not have updated successfully, the final step in the deployment process is very important-auditing systems to verify the success of the deployment. In some organizations, auditing will be performed by a separate auditing group, often connected with security teams. Microsoft tools such as WSUS, MBSA, and SMS can be used to audit for the successful application of security updates. In addition, there is information in the security bulletin in the Security Update Information section on how to verify the successful application of the security update.
Conclusion
With the Microsoft monthly security bulletin release, it's possible for you to implement more regular planning and processes for the management of security updates. Building regular procedures and processes helps you better protect your systems while at the same time minimizing disruption and possible downtime.
While every organization should build its own unique processes for managing security updates, your organization's processes should include the steps recommended in this column. You don't need to have separate security and update teams, but you should perform both a security risk assessment and develop a deployment plan.
The key concept here is the importance of planning. With good planning, the steps that involve actual activity, such as the testing and deployment stages, will go much more smoothly, which, in turn, will result in a successful security update process.
Christopher Budd is a security program manager in the Microsoft Security Response Center (MSRC). He specializes in technical communication with IT and security professionals and other software users.
© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.