Microsoft IT Tests Line of Business Application Compatibility for Internet Explorer 9
Published: March 2011
The following content may no longer reflect Microsoft’s current position or infrastructure. This content should be viewed as reference documentation only, to inform IT business decisions within your own company or organization.
With the prerelease deployments of Windows Internet Explorer 9, Microsoft IT (MSIT) deepened and enhanced its extensive application compatibility testing process. MSIT invested in more testing cycles while reducing the resource hours required to configure test machines by taking advantage of Virtual Machine efficiencies in the lab environment. These improvements enabled MSIT to better serve engineering teams, to validate more applications, and to deliver strong application backwards compatibility.
Products & Technologies
Article, 112 KB, Microsoft Word file
IT professionals who want to understand how Internet Explorer 9 can improve their operations and deployment personnel who want to learn how Microsoft IT minimized the impact that the upgrade process had on operations through use of preconfigured virtual machines.
March 2011 brought the release of Microsoft Windows Internet Explorer 9. Internet Explorer 9 delivers robust backwards compatibility, along with many new features designed to take advantage of emerging Web standards. Continued support of the Compatibility View feature ensures that applications developed to Internet Explorer 7 standards will continue to work well with Internet Explorer 9.
Microsoft IT supports the technology infrastructure across the Microsoft enterprise. As its First and Best customer, Microsoft IT has been closely involved in the Internet Explorer 9 development process for over a year. MSIT managed the compatibility testing of pre-beta, beta, release candidate, and final versions of Internet Explorer 9 against a selection of internal line of business (LOB) applications used by teams at Microsoft. The process enables MSIT to identify and resolve product bugs early in the development process. Validating the success of the testing process, issues related to LOB applications have accounted for less than 0.5% of the Helpdesk call volume throughout the internal prerelease deployment of Internet Explorer 9.
This article provides an overview of the compatibility testing process. Deployment best practices demonstrate how MSIT enhanced process efficiencies by leveraging desktop virtualization. Some organizations may discover that the centralized test process, using virtualization technology Hyper-V™, offers an increase in efficiencies for testing LOB application compatibility in advance of a larger scale deployment for Internet Explorer 9. Note that this document and its recommendations should not be considered prescriptive, but should be modified appropriately for each company's specific requirements and implementation processes.
Line of Business Application Compatibility Testing
Like any other IT organization, MSIT must ensure baseline application compatibility for business critical applications. Internet Explorer 9 is no exception. MSIT is responsible for hundreds of LOB applications used within Microsoft. The majority are developed and hosted internally. Others developed by third party vendors may be hosted internally or externally. At Microsoft, internal prerelease software deployments are referred to as dogfood testing. The dogfood process exposes issues that other enterprise customers may eventually encounter. This allows Microsoft to either fix or find workarounds for any issues that are discovered. During the early adoption process, MSIT tests a group of the most critical LOB applications prior to each product milestone release. For more information about the LOB application testing methodology used by Microsoft IT, see http://download.microsoft.com/download/4/e/8/4e8c1b99-54f1-4be7-8e9b-78024015848d/LOBApplicationCompatibilityTesting_TWP.doc.
Testing Process Overview
The first step Microsoft IT took was to review the list of key LOB applications that were used to test previous releases of Internet Explorer. Then, applications were assessed according to two criteria. Those applications considered to be of critical importance for business continuity were identified and Helpdesk impact was evaluated. Applications with a large and active user base were also identified because impact to those applications had the potential to result in a high volume of issues reported to Helpdesk.
MSIT tested prerelease versions of Internet Explorer 9 to cover the most common internal and external configurations. This included Windows Vista® SP2, Windows 7, and a prerelease version of Windows 7 SP1. Since February 2010, seven formal one-week test passes were conducted. Subsequent testing to verify that identified issues had been resolved also ensured that no new issues were introduced with the fixes. While the focus centered on the key applications that require testing, MSIT also encouraged a high level of voluntary participation as schedules allow. Voluntary testing accounts for approximately two-thirds of the overall testing performed on each product which provides extremely valuable feedback to Microsoft product groups.
With the release of Internet Explorer 9, Microsoft IT proactively increased application compatibility. Five testing cycles were committed to prerelease versions of Internet Explorer 8. With Internet Explorer 9, the investment in testing was increased to seven cycles. By conducting additional test passes, MSIT was able to validate more LOB applications in preparation for the companywide deployment. For example, two comparable cycles included the following number of internal applications:
- 129 for the Internet Explorer 8 Beta 2
- 170 for the Internet Explorer 9 Beta
The increases in testing cycles and the number of applications tested demonstrate the commitment of Microsoft IT and the Internet Explorer product group to releasing Internet Explorer 9 with the highest possible quality.
Finding New Efficiencies
Microsoft IT uses virtual machines (VMs) to address a core efficiency issue in application compatibility testing. Testing on client hardware from different manufacturers and models introduced too many unrelated variables in the testing environment. To resolve that problem, testing environment standardization was critical and VMs are the most cost-effective method of standardizing the environment. Using VMs allowed system engineers to run the latest prerelease version of Internet Explorer 9 on a virtual desktop computer.
The ability to run LOB applications on virtual desktops that already have a working operating system in place provides a higher level of control and efficiency than building a similar system on a physical desktop. When issues are found, they are attributable only to the new software because the team can verify that the virtual desktop worked properly before the addition occurred.
Microsoft IT created a lab to host only the VMs used for LOB application compatibility testing. MSIT leveraged recent advances in desktop virtualization and virtualization management available in Windows Server® 2008 R2 and Microsoft System Center to achieve greater efficiencies. Creating and managing pools of VMs for various application compatibility testing scenarios reduced support calls for unrelated issues. Providing preconfigured VMs to MSIT engineering groups increases the speed of the testing process. The groups can now test LOB applications immediately, rather than using valuable time to reconfigure their personal workstations to produce a temporary environment.
The Microsoft IT LOB application compatibility lab hosts multiple servers which in this environment provide a large number of VMs to all testers who wish to participate in the test passes. Virtual desktops work well for application testing because an administrator can quickly restore them to a specified state. The VMs can then be reissued quickly to other testers in times of high demand. One physical server can host numerous VM images, and Microsoft IT has found that the optimum performance-to-density ratio is 10 VMs per instance of Windows Server 2008 R2 server and Hyper-V.
System Center Virtual Machine Manager
MSIT uses System Center Virtual Machine Manager 2008 R2 (SCVMM) to manage the VM infrastructure, including the VMs and the software images for each test pass. Using SCVMM to manage the Hyper-V servers means that MSIT can:
- Maintain a library of templates for specific client configurations. This accelerates the testing process by allowing prerelease software installation onto multiple virtual desktop configurations simultaneously. The specific configuration can then be cloned to produce the number of VMs required.
- Monitor the VMs during the test pass from a single interface. From this interface, virtual hardware changes can be made, machines can be stopped and restarted, and additional software can be installed upon request.
- Store a library of VM snapshots from previous test passes. The snapshots can be redeployed later to investigate issues discovered during testing, or to validate product group fixes for those issues.
The MSIT application compatibility testing lab consists of 20 servers. By having more than 100 preconfigured VMs available, testers began running prerelease applications on a virtual desktop as soon as they were assigned. The testers avoided hours of configuring physical PCs to create test environments. Each VM saves approximately four to five hours, per tester, that would normally be spent on test environment configuration tasks. By using VMs, MSIT was able to test over 200 of the most complex internal LOB applications in one week. To date, the highest number of applications tested in a single week has reached 245. This included voluntary participation, beyond the list of targeted critical applications.
Microsoft IT spent over seven months testing LOB application compatibility of prerelease builds of Internet Explorer 9 before the beta version was made available to the public. The prerelease test passes gave Microsoft IT insight as to which bugs were fixed with the beta release and how to resolve the types of issues that were likely to trigger a spike in internal help desk calls.
Microsoft maintains an intranet site to share LOB application testing information. MSIT maintains a list of known issues, testing schedules, and the most current list of applications included for LOB test passes. The site also publishes user documentation that accompanies specific LOB prerelease applications, along with any known workarounds for known issues.
As a result of this proactive approach, MSIT communicated early and often with application owners regarding issues. The ability to share information about functionality issues enabled application owners to make adjustments before user impact translated to Helpdesk calls. For example, the site conveyed that a browser user agent string change may need to be applied.
Browser Query String Changes
The user agent string identifies the type and version of the web browser in use. As with previous versions of Internet Explorer, changes to the browser user agent string can introduce issues with pages that parse the string in script. Before the first test pass, the LOB application owners involved in the Microsoft IT testing were advised of the issue and were provided a resolution method.
Updates to Compatibility Mode
Internet Explorer 9 preserves the backwards compatibility modes that also were available in Internet Explorer 8. The default mode for intranet is set to Compatibility View, which defaults to the Internet Explorer 7 document mode and user-agent string. For more information, see http://msdn.microsoft.com/library/cc288325(VS.85).aspx. This flexibility allows for multiple methods to address compatibility issues. For example, when deploying Internet Explorer 9, MSIT encountered minor page display issues for LOB applications. Specifically, alignment, text, and controls spacing issues may occur if an application was built to adhere to older user interface and layout standards designed for earlier versions of Internet Explorer.
The solution for these issues, particularly for any application that rendered correctly in Compatibility View in Internet Explorer 8, is to configure Internet Explorer 9 to render application pages in Compatibility View. MSIT has used a number of the methods now built in to Internet Explorer to facilitate this process. For example:
- MSIT enables Compatibility View for the Intranet Zone as a default setting. This ensures the highest level of compatibility with internal applications and web sites.
- MSIT then leverages Group Policy to populate the Compatibility View list with specific external URLs, ensuring compatibility with those sites as well.
Ongoing Issue Resolution
An ongoing part of the testing process involves the review of reported issues to determine which apply to the browser itself, versus those that are related to the underlying application.
Prerelease testing revealed some issues where pages were not rendering as designed because an application was designed to the standards of an earlier version of Internet Explorer. Typically, minor changes in HTML resolved the issue and a page rendered properly. MSIT shared these issues and the suggested fixes to application owners as testing continued.
To date, LOB testing for prerelease versions of Internet Explorer 9 shows improved overall pass rates when compared to analogous prerelease versions of Internet Explorer 8. One contributing factor to the improved testing emerges from increased voluntary application testing for Internet Explorer 9. This provided invaluable feedback to the product group and MSIT. The two versions of Internet Explorer 8 that preceded the final release candidate each had an overall pass rate of 93%. The pass rates for the two most recently completed test passes for Internet Explorer 9 Beta and for the initial release candidate, are 95% and 97% respectively.
Many of the best practices identified during the LOB application compatibility testing for Internet Explorer 8 still apply. This section revisits those guidelines and includes updates on how using Microsoft technology can bring more efficiency to the process of testing application compatibility in general.
Web developers and IT Pros who build, test, and maintain Web applications for Internet Explorer 9 can benefit from these practices:
- Use the Internet Explorer 7 group policy list and Quirks mode group policy list to ensure that critical Internet or extranet LOB sites render properly. This is particularly helpful for sites hosted by 3rd party vendors.
- Include the X-UA-Compatible tag to ensure your Web sites render properly in current and future versions of Web browsers. Consider adding the tag to sites when creating new applications, or when making changes during a site revision. The meta tag is also helpful for intranet LOB sites that do not want to render in Compatibility View by default.
- Use semantic, structured XHTML markup code to create standards-based Web sites that are scalable, available, and interoperable.
IT Pros who plan and conduct application compatibility testing can benefit from these practices:
- Identify and maintain a centralized list of the LOB applications considered critical for business continuity. Also, consider tracking complex applications with a substantial user base. Having such updated lists aids the testing process by reducing the overall planning effort.
- Allow volunteer testing of LOB applications that may not be business critical, but can benefit from a centralized test process.
- Maintain a database of historical compatibility data for the LOB applications. This information enables easier identification of those applications likely to experience issues before deployment of a newer version of a product. These applications should be tested in addition to those required for business continuance.
- Take advantage of virtualization technologies:
- Preconfiguring VM clients through Windows Server 2008 R2, and using Hyper-V to host the preconfigured VM clients significantly reduces the complexity of hardware management. It eliminates the need to maintain multiple client computers of varying ages through the hardware lifecycle, and allows IT Pros to focus on testing on a small set of servers.
- Assign, manage, and recycle VMs through SCVMM. This management tool enables you to increase the speed of testing by reducing the time it takes for test teams to get started.
- Apply the principle of isolation when testing application compatibility. Test only one update, whether it is an operating system or a Web browser change. By doing this, the process of identifying root issues is simplified.
- Communicate clearly with users about known issues with deploying changes and the reasons for software updates. Convey whether changes are required for security or business reasons, or merely suggested. LOB application testing may identify issues that cannot be solved before deployment occurs. Work closely with application owners to develop workarounds or to identify alternatives. Include that information in a communication campaign when it is made available to users.
Thanks to an increase in prerelease application compatibility testing cycles and improvements in the testing process itself involving the use of virtualization technologies, Microsoft IT helped the product group deliver high quality for Compatibility Mode in Internet Explorer 9. Both teams are committed to the goal that if a LOB application worked with Internet Explorer 8, it should work with Internet Explorer 9.
Communicating known issues and workarounds to application owners prior to testing resulted in lower Helpdesk call volumes and also kept the testing program's focus on new issues. Use of preconfigured VMs created new efficiencies in the testing process that enabled more testing to occur in a timeframe similar to that used for prerelease testing for Internet Explorer 8.
The MSIT experience reinforced lessons learned from previous dogfood efforts, and saw significant new gains in efficiencies and cost reductions from using publicly available versions of Windows Server 2008 R2, Hyper-V, and SCVMM.
For More Information
For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:
© 2011 Microsoft Corporation. All rights reserved.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Windows, Hyper-V, and Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.