Migrating a Large, High-Volume Web Site to Internet Information Services 7.0

Technical White Paper

Published: April 11, 2008

On This Page

Executive Summary
Introduction
Business Benefits
Organization
Background
Planning the Migration
Building the Necessary Components and Solutions
Deploying in the Production Environment
Operating Microsoft.com After Migration
Planning for Future Improvements
Best Practices
Conclusion
For More Information
Appendix A: Tools Used in Testing and Deployment Processes

Executive Summary

Managing and administering a site like Microsoft.com requires consistent attention and a high degree of automation. The Microsoft.com Operations team is always looking for methods to reduce the manual effort required to monitor, manage, and administer the many sites hosted on Microsoft.com and the server farm running Microsoft.com. One of the reasons why Microsoft deployed Internet Information Services (IIS) version 7.0 is that new features in IIS 7.0 can help reduce that effort.

Microsoft also deploys pre-release versions of Microsoft products and software updates (such as service packs) on Microsoft.com to help ensure that the products work well in a large-scale environment prior to release. Microsoft.com Operations has developed processes for performing these deployments with minimal interruption. This helps ensure that Microsoft.com is one of the most highly available sites on the Internet.

This white paper describes how Microsoft.com migrated more than 80 Web servers, managing more than 5 gigabytes (GB) of content in more than 750 virtual roots, to IIS 7.0. Not only did the Microsoft.com Operations team reduce the effort to manage and administer the site, but it also realized a 10 percent performance gain.

This white paper covers the:

• Business benefits achieved in deploying IIS 7.0 on Windows Server® 2008.
• Teams at Microsoft that were involved in the development, deployment, and ongoing management of the solution.
• Background on Microsoft.com, including the products and technologies used in creating the solution.
• Process that Microsoft.com Operations followed during the planning, building, deploying, and operating phases of the migration from IIS version 6.0 to IIS 7.0.
• Deployment of IIS 7.0 running on Windows Server 2008 in the Internet-facing Microsoft production environment.
• Operation of Microsoft.com after the migration to IIS 7.0.
• IIS 7.0 features and improvements that the Microsoft.com Operations team will implement in the future.
• The best practices that Microsoft.com Operations used in migrating to IIS 7.0 and Windows Server 2008.

This document shares the experiences of Microsoft teams in migrating Microsoft.com from IIS 6.0 running on Windows Server 2003 to IIS 7.0 running on Windows Server 2008. Because of the significant amount of knowledge that these teams gained, the experience provides relevant guidance to organizations that want to reduce downtime during migration, reduce ongoing operations effort, and take advantage of the financial investment in digital assets by deploying IIS 7.0 and Windows Server 2008.

This white paper assumes that readers are technical decision makers and are already familiar with IIS, Windows Server, and Microsoft® SQL Server®. Many of the principles and techniques that this paper describes can be employed to manage risk within any organization, and the design considerations for remote access infrastructure can likewise apply to most enterprise-scale IT environments through Microsoft products. However, this paper is based on the experience and recommendations of the Microsoft.com Operations teams as an early adopter. It is not intended to serve as a procedural guide. Each enterprise environment has unique circumstances; therefore, each organization should adapt the plans and lessons learned described in this paper to meet its specific needs.

Introduction

Microsoft.com is an asset for current and potential Microsoft customers that enables them to get more information about Microsoft products and services. The information helps customers make informed business decisions 24 hours a day, 7 days a week. This "around-the-clock" information is important because Microsoft.com serves a global audience that relies on Microsoft.com no matter what time of the day or what day of the week.

In addition, the Microsoft.com site is a portal to other key IT professional and developer sites such as TechNet, MSDN® (the Microsoft Developer Network), Communities, and the Download Center. These key sites enable customers to obtain the latest hotfixes, service packs, knowledge base articles, and online product documentation. IT professionals have come to rely on Microsoft.com to help them plan, build, deploy, and operate Microsoft products.

Because current and potential customers rely heavily on Microsoft.com, the site is mission critical for Microsoft. Any outages or interruption of services for Microsoft.com potentially affects these customers, reduces confidence in Microsoft products, and reduces the company's revenue. Therefore, when Microsoft.com Operations performs any configuration changes or updates to Microsoft.com, it must ensure that the changes or updates do not affect the availability of the site.

The Microsoft.com Operations team frequently coordinates with internal Microsoft product teams to deploy prerelease versions of the Windows Server operating system, IIS, and the .NET Framework. Although this is not typical of other organizations, Microsoft.com Operations deploys prerelease software versions to help ensure Microsoft products are tested early on in the software life cycle in a real-world production enterprise environment.

The Microsoft product teams value this relationship because it enables them to get real-world feedback and improve the reliability and scalability of their products. The Microsoft.com Operations team values the relationship so that during the early adoption of pre-release Microsoft software they can provide feedback on the key features to be included in the next versions of products and help meet their current and future business needs.

Another advantage to deploying prerelease versions of Microsoft software is that the Microsoft.com Operations team has gained significant experience in performing these deployments without affecting the availability of the site. The disadvantage to deploying all these versions of software is that the Microsoft.com Operations team is constantly updating the site and creating a lot of administrative and management overhead.

The Microsoft.com Operations team closely monitors and manages the Microsoft.com site to help reduce any outages, interruptions in service, or performance degradation. In addition to providing real-world testing of IIS 7.0 and Windows Server 2008, the Microsoft.com Operations team wants to take advantage of the new monitoring and management features in IIS 7.0 and Windows Server 2008.

Business Benefits

Like any other organization, Microsoft needs to realize a bottom-line benefit to the day-to-day business functions. Some of these benefits are objective, such as the financial savings in the effort required to support the solution and the improvements in administrative and management efficiency. Other benefits are subjective, such as customer satisfaction with the solution.

Microsoft gained the following benefits when it migrated Microsoft.com to IIS 7.0 and Windows Server 2008:

• Reduced administrative effort to support the solution
• Reduced effort to migrate sites to IIS 7.0 and Windows Server 2008
• Improved performance
• Improved security of the solution
• Improved monitoring and identification of potential problems with the solution

Reduced Administrative Effort to Support the Solution

One of the design goals for IIS 7.0 is to reduce the amount of administrative effort required to support Web sites and applications. One of the primary reasons why Microsoft upgraded to IIS 7.0 was to take advantage of these new features.

Migrating to IIS 7.0 reduced the administrative effort by:

• Providing an improved configuration method. The applicationHost.config file is an .xml format file that contains the IIS 7.0 configuration settings. Because the file is in .xml format, it is easier to modify than the metabase in IIS 6.0 and is easier than making changes to the registry. Also, all of the configuration settings are stored in this file instead of being stored in multiple places in IIS 6.0 (such as the metabase, web.config file, or registry).
• Supporting command-line-based deployments. IIS 7.0 includes a command-line-based deployment method that supports the automated installation of individual IIS 7.0 roles and features by using the ServerManagerCmd.exe tool. This method enables a fully automated installation of IIS 7.0 when an administrator is deploying a new Web server or is refreshing an existing Web server.
• Taking advantage of the improvements in the new modular IIS 7.0 management console. The IIS 7.0 management console is more intuitive and task oriented than the management console in IIS 6.0.
• Providing delegation of IIS 7.0 administration. An administrator can now delegate the administration of the IIS configuration settings, such as HttpErrors and HttpRedirect configuration settings, to application owners. This flexibility in configuration distributes the administrative effort and reduces the dependency on one individual or team to perform all administrative tasks.
• Including support for Windows PowerShell™ scripting. Windows PowerShell is a new command-line shell and scripting language that helps IT professionals achieve greater productivity and control system administration more easily. Windows PowerShell contains more than 130 standard command-line tools, a new administrator-focused scripting language, and consistent syntax and tools. It accelerates automation of system administration tasks in IIS 7.0. Based on their experience, Microsoft.com Operations team members found Windows PowerShell to be easy to adopt, learn, and use, because it does not require a background in programming, and it works with existing IT infrastructures, existing scripts, and existing command-line tools.

Reduced Effort to Perform Migration

Microsoft.com Operations has deployed all of the released versions of Windows Server 2008 and IIS 7.0 on Microsoft.com and many internal builds. In fact, it started running live traffic to a single computer running IIS 7.0 and Windows Server 2008 in May 2005. Because of the0020sheer volume of traffic on the site and the frequency that new versions of IIS 7.0 builds are deployed, the team needed to minimize any effort required to perform the migration.

Microsoft.com Operations reduced the effort to perform the migration to IIS 7.0 by:

• Taking advantage of the high degree of compatibility with existing applications. The majority of existing Microsoft ASP.NET-based applications run in IIS 7.0 in integrated mode without modifications. Integrated mode is the new application pool processing model in IIS 7.0. An application pool is a process boundary that separates worker processes from each other. This helps ensure that a Web site or application running in one application pool will not be affected by application problems in other application pools. Application pools significantly increase both the reliability and manageability of a Web infrastructure.
• Configuring IIS 7.0 to be compatible with existing applications. Fewer than 2 percent of the existing ASP.NET-based applications were incompatible when running in integrated mode. Microsoft.com Operations ran these incompatible applications in a default application pool configured to run in classic mode. Classic mode enables code to run in application pools as it did in IIS 6.0. When running in classic mode, ASP.NET is connected to IIS as a stand-alone application framework through the Internet Server Application Programming Interface (ISAPI) model.
• Configuring IIS 7.0 to be compatible with existing installation and administration scripts. The configuration system in IIS 7.0 has undergone significant changes, moving away from a centralized, loosely typed configuration store (metabase in IIS 6.0) to a delegated XML configuration file hierarchy (applicationHost.config file in IIS 7.0). IIS 7.0 provides the IIS 6.0 Management Compatibility components as an emulation layer for the metabase. The IIS 6.0 Management Compatibility components support code that uses Admin Base Object (ABO) application programming interfaces (APIs), higher-level Windows® Management Instrumentation (WMI) APIs, or Active Directory® Service Interfaces (ADSI) APIs. Microsoft.com Operations has a number of existing scripts that configure and manage Microsoft.com. The IIS 6.0 Management Compatibility components enable these exiting scripts to run with minimal modifications, but the existing scripts now write to the new configuration system by using the ABO translation layer.
• Using tools to help migrate applications to integrated mode. Microsoft.com Operations moved the existing ASP.NET-based applications that did not run initially in integrated mode to an application pool running in classic mode. Then, Microsoft.com Operations ran the Appcmd.exe command-line tool with the Migrate switch to identify configuration changes that could help migrate the application to run in integrated mode. Finally, Microsoft.com Operations coordinated with the application owners and content providers to update the appropriate configuration files and moved the ASP.NET applications into an application pool running in integrated mode. After the team completed this process, fewer than 2 percent of the existing applications remained in the application pool running in classic mode.

Improved Performance

During the early stages of adoption, Microsoft.com Operations goals for deploying IIS 7.0 and Windows Server 2008 predominantly focused on improving administration and management. However, the released version of IIS 7.0 and Windows Server 2008 also resulted in a 10 percent performance gain compared to IIS 6.0 and Windows Server 2003.

Microsoft.com Operations obtained these results by measuring the performance of Microsoft.com running IIS 6.0 on Windows Server 2003 and IIS 7.0 running on Windows Server 2008. The team closely monitored the performance over a 72-hour period for both test cases.

The chart in Figure 1 illustrates the performance that the team gained by running IIS 7.0 and Windows Server 2008 on the same computer resources. IIS 7.0 and Windows Server 2008 are capable of supporting 10 percent more requests per second than IIS 6.0 and Windows Server 2003. This improvement in performance translates to Microsoft.com processing more than an additional 100 million requests per day at the same processor utilization.

Figure 1. Microsoft.com performance test results

Improved Security

Security is a top priority for any Internet presence like Microsoft.com. IIS 7.0 includes many security-related improvements compared to previous versions of IIS. Microsoft.com Operations improved the security of Microsoft.com by:

• Reducing the attack surface. IIS 7.0 provides detailed control over installing its features and components. Installing fewer role services reduces the attack surface of the computer running IIS.
• Incorporating security features. In IIS 6.0, additional features and technologies improved security (such as URLScan). These security-related technologies have been integrated into IIS 7.0 and can be configured in the applicationHost.config file. This tighter integration provides improved security and ease of configuration.

Improved Monitoring and Problem Identification

For sites as large as Microsoft.com, real-time monitoring and root-cause identification of problems are an important requirement. Monitoring and problem identification are essential to helping ensure high uptime and optimal performance.

Migrating to IIS 7.0 and Windows Server 2008 improved monitoring and problem identification by:

• Providing improved diagnostics and monitoring of Web sites. The Failed Request Tracing feature enables the Microsoft.com Operations team to capture a detailed trace of Hypertext Transfer Protocol (HTTP) requests in a XML-formatted log file. An administrator can configure the rules that determine which requests should be captured in the log by specifying parameters (such as file extension type, HTTP status code, time take, and unique URL) This feature helps administrators save time by not having to reproduce the problem before working on a solution. Administrators can also use the Appcmd.exe tool to view real-time information about current running HTTP requests.
• Automating responses to events. An improved task scheduler in Windows Server 2008 can initiate tasks based on specified events. The tasks can automate recovery processes or send alert notification of an event.

Organization

The Microsoft teams that participated in the early adoption of Windows Server 2008 and IIS 7.0 on the Microsoft.com site included the teams related to Microsoft.com and the IIS product team.

Microsoft.com Teams

The following teams related to Microsoft.com have specific responsibilities in managing and providing content for the Microsoft.com site:

• Microsoft.com Operations team
• Microsoft.com Application and Content Providers

Microsoft.com Operations Team

The Microsoft.com Operations team runs sites such as Microsoft.com, MSDN, and Microsoft Update that have high volumes of traffic due to the large demand by the Microsoft customer base. The team manages more than 120 different Web sites hosted on more than 2,000 production servers. The infrastructure to run these sites provides an excellent opportunity for real-world testing of technologies.

The Microsoft.com Operations team consists of 60 members, divided into the following teams:

• Microsoft.com Operations Tier 1 and 2. These teams monitor the systems and handle the initial site incidents and customer service requests. They are the first line of defense for Tier 3 engineers.
• Microsoft.com Operations Tier 3. This team consists of Web and database engineers who support and maintain the server infrastructure, incident management, system upgrades, patch management, and application deployments on Microsoft.com Web properties.
• Microsoft.com Debug. This team is the primary escalation point for the Microsoft.com Operations Tier 3 team, should a complex issue arise that requires system debug or product team support.
• Product Adoption and Evangelism. This team is dedicated to the product adoption and evangelism of operational best practices by publishing white papers, contributing to blogs, contributing to the online TechCenter, and conducting face-to-face customer engagements.
• Security and Architecture team. This team is responsible for core infrastructure, security, and monitoring technologies required to run the Microsoft.com properties.

Microsoft.com Application and Content Providers

Microsoft.com has centralized operations management, but it has distributed teams that create and manage the applications and content hosted on the site. Hundreds of internal Microsoft marketing staff, service content providers, and product teams design, author, and publish the applications and content for the Microsoft.com site. The application and content providers design, update, and sustain the site for optimal user experience, including information architecture, navigation, branding, content management and programming, publishing, and hosted services. Based on traffic data, user feedback, and industry trends, the team develops and prioritizes business requirements for site features and functionalities, working with the other teams through technical implementation and deployment.

IIS Product Team

The IIS product team provided technical guidance to early adopters, such as the Microsoft.com Operations team on the IIS component, and was the primary point of escalation for the Microsoft.com Operations team. During this period of early adoption, the IIS product team worked with the Microsoft.com Operations team and other adopters to refine the deployment and operations documentation. This was a huge benefit to the Microsoft.com Operations team. In return, the Microsoft.com Operations team provided a platform to validate the IIS 7.0 component in a real-world enterprise implementation of IIS 7.0 in a production environment. The teams established shared goals early in the adoption process.

The Microsoft.com Operations team provided feedback to the IIS product team based on issues experienced during the adoption process. This feedback often resulted in new features or enhancements in IIS that help ensure that the component is ready to deploy in enterprise environments.

Background

When Microsoft.com started in 1994, the entire site was hosted on a single server that handled about 1 million hits a day. As the Internet expanded, more and more groups at Microsoft began adding content to the Web site. The number of computers and network infrastructure expanded accordingly. Table 1 lists the current characteristics of Microsoft.com in terms of traffic, content, Web server configuration, and hardware.

Table 1. Characteristics of Microsoft.com

Planning the Migration

Before Microsoft.com Operations deploys any new products, service packs, critical updates, or hotfixes in its production environment, it does extensive planning and testing. The first step in this process is to perform all planning-related tasks. The migration to IIS 7.0 and Windows Server 2008 was no exception to this process.

Identifying and Analyzing Web Applications

Planning the migration to IIS 7.0 included identifying and analyzing all the Web applications running on the existing solution. In any migration, application compatibility with the new solution is one of the major concerns. To perform this step, Microsoft.com Operations:

1. Created an inventory of the applications running on Microsoft.com.
2. Analyzed each application to determine compatibility with IIS 7.0 integrated mode.
3. Analyzed each installation or management script for each application to determine whether the IIS 6.0 Management Compatibility components were needed

Microsoft.com used this application inventory later in the process to help ensure that all compatibility issues had been resolved.

Selecting the Compatibility Features

IIS 7.0 includes many features to help applications written for IIS 6.0 to run on IIS 7.0. Microsoft.com Operations used the information from the application inventory to determine which features it needed for the existing applications and scripts. The compatibility issues can be divided into the following categories:

• Existing application compatibility with application pools configured for integrated mode
• Installation and administration software requirements for IIS 6.0 management interfaces

Determining Application Compatibility with Integrated Mode

Application pools set boundaries for the applications they contain, which means that any applications that are running outside a specified application pool cannot affect the applications in the application pool.

In IIS 7.0, application pools run in either integrated mode or classic mode. The application pool mode affects how the server processes requests for managed code. If a managed application runs in an application pool with integrated mode, the server will use the integrated, request-processing pipelines of IIS and ASP.NET to process the request. However, if a managed application runs in an application pool with classic mode, the server will continue to route requests for managed code through Aspnet_isapi.dll, processing requests the same as if the application were running in IIS 6.0.

Integrated mode and classic mode are configured for each application pool running on a server. Because these settings are for each application pool, an administrator can run some application pools in integrated mode and other application pools in classic mode on the same server.

Most managed applications should run successfully in application pools with integrated mode. However, some applications may need to run in classic mode for compatibility reasons.

Microsoft.com Operations determined application compatibility mode by performing the following steps for each application:

1. Run an application in integrated mode to determine compatibility without any modifications. The majority of existing applications running on Microsoft.com ran in integrated mode without modification. During the Microsoft.com migration process, less than 2 percent of the applications had a breaking change.
2. If the application does not run without modification, migrate the application configuration. Some of the existing applications required the configuration to be migrated so that the applications would be compatible with integrated mode. Microsoft.com Operations migrated the application configuration by using the Appcmd.exe tool with the migrate config option. The basic format of the migration command is as follows:

   %windir%\system32\inetsrv\APPCMD.EXE migrate config Application Path

   For more information about migrating application configuration by using Appcmd.exe, refer to "Getting Started with AppCmd.exe" at http://www.iis.net/articles/view.aspx/IIS7/Use-IIS7-Administration-Tools/Using-the-Command-Line/Getting-Started-with-AppCmd-exe.

3. Run the application again in integrated mode to determine whether migrating the application configuration resolved the compatibility issue. After migrating the application configuration, Microsoft.com Operations retested the application in integrated mode. Again, fewer than 2 percent of the ASP.NET applications were incompatible after the team migrated the application configuration.

   The compatibility issues that Microsoft.com Operations encountered with the ASP.NET applications that were incompatible with integrated mode occurred during application initialization. Specifically, the HttpContext.Request, HttpContext.Response, HttpApplication.Request, and HttpApplication.Response properties were not available when the Application_Start event was invoked, and when the HttpModule.Init method was called the first time. To resolve this compatibility issue, Microsoft.com application owners needed to modify the application code for the incompatible application.

   For some of the applications, Microsoft.com Operations worked with the application owners and content providers to modify the code to be compatible with integrated mode. The remaining incompatible applications are running in an application pool in classic mode.

4. If the application does not run after migrating the application configuration, run the application in classic mode. For the application that had a breaking change, Microsoft.com Operations ran that application in an application pool configured for classic mode. Long term, Microsoft.com Operations will coordinate with the application owner to revise the application to be compatible with integrated mode. Running an application in classic mode should be viewed as a short-term solution.

For more information about breaking changes for applications running in integrated mode on IIS 7.0, refer to "Breaking Changes for ASP.NET 2.0 Applications Running in Integrated Mode on IIS 7.0" at http://mvolo.com/blogs/serverside/archive/2007/12/08/IIS-7.0-Breaking-Changes-ASP.NET-2.0-applications-Integrated-mode.aspx.

Determining Installation and Administration Script Requirements for IIS 6.0 Compatibility

One of the improvements in IIS 7.0 is the configuration model that the applicationHost.config file provides. IIS 7.0 uses this file instead of configuration settings that IIS 6.0 previously stored in .config files, the registry, and the metabase.

However, existing installation and administration scripts might have dependencies on the configuration interfaces that IIS 6.0 supported, such as the ABO interface or the ADSI and WMI providers that are based on ABO. Also, other applications may expect the IIS 6.0 management console to be installed to function properly. To provide backward compatibility, IIS 7.0 includes the IIS 6.0 Management Compatibility components. These components include:

• Metabase Compatibility. This component provides compatibility for the configuration interfaces supported in IIS 6.0 (such as the ABO interface, ADSI provider, and WMI provider). Only the root IIS configuration file, the configuration settings supported by IIS 6.0, is supported. The Microsoft .NET Framework and new IIS 7.0 configuration settings are not exposed to these interfaces.
• Management Console. This component provides compatibility for applications that require the IIS 6.0 management console.

Microsoft.com Operations initially used the IIS 6.0 Management Compatibility components to enable existing installation and administration scripts to be compatible with IIS 7.0. However, long term, the team will migrate the existing software to the IIS 7.0-supported interfaces. After all the IIS 6.0 compatibility requirements are eliminated, the team will uninstall the IIS 6.0 Management Compatibility components.

For more information about compatibility with IIS 6.0 management interfaces, refer to:

• "Metabase Compatibility with IIS 7.0" at http://www.iis.net/articles/view.aspx/IIS7/Use-IIS7-Administration-Tools/Using-XML-Configuration/Metabase-Compatibility-with-IIS7.
• "Deep Dive into IIS 7.0 Configuration" at http://www.iis.net/articles/view.aspx/IIS7/Use-IIS7-Administration-Tools/Using-XML-Configuration/Deep-Dive-into-IIS7-Configuration.
• "How to Use Metabase Compatibility with IIS 7.0" at http://www.iis.net/articles/view.aspx/IIS7/Use-IIS7-Administration-Tools/Using-XML-Configuration/How-to-use-Metabase-Compatibility-with-IIS7.

Building the Necessary Components and Solutions

After it completed the planning-related tasks, Microsoft.com Operations started the build phase. During the build phase, Microsoft.com Operations made all the necessary modifications to the Web-based applications and deployment and administration scripts. In addition, during this phase, Microsoft.com Operations performed more extensive testing of the solution, including a pilot deployment in the production environment.

Performing Modifications to Web Applications

To modify Web-based applications, Microsoft.com Operations ran the Appcmd.exe tool with the migrate config option for the applications that did not run in integrated mode. The team ran the Appcmd.exe tool first in a preproduction environment that closely mirrored the production environment for Microsoft.com. Later in the process, it ran the Appcmd.exe tool in a limited pilot deployment in the production environment.

Performing Modifications to Deployment and Administration Scripts

Microsoft.com Operations has developed mature deployment and administration processes over the years of deploying, maintaining, monitoring, and managing Microsoft.com. This process consists of using well-planned release processes and custom-developed scripts.

Most of the scripts worked without modifications when Microsoft.com Operations deployed the IIS 6.0 Management Compatibility components. However, many of the scripts made configuration changes that were not necessary for IIS 7.0 or needed to be changed for how IIS 7.0 is configured. The script changes included:

• Grant permissions to the new built-in IUSR account. A new Windows built-in group named IIS_IUSRS replaces the local IIS_WPG group. A new Windows built-in account called IUSR replaces the local IUSR_MachineName anonymous account from IIS 6.0. IIS 7.0 uses these new built-in group and user accounts for anonymous access to Web sites. However, IIS 7.0 continues to use the IUSR_MachineName account for anonymous access to FTP sites. Microsoft.com Operations changed the scripts so that permissions are assigned to the new user and group accounts.
• Remove the IIS-related configuration settings. The build script for Microsoft.com modified the metabase in IIS 6.0. The IIS settings are now configured in the applicationHost.config file, which contains the ASP.NET and IIS-related configuration settings. Microsoft.com Operations modified the build script to make the appropriate changes to the applicationHost.config file. Microsoft.com Operations did not remove the Simple Mail Transfer Protocol (SMTP)-related settings in the build script, because these settings are still configured in the metabase.
• Remove the URLScan-related configuration settings. Microsoft.com Operations removed the URLScan ISAPI filter and the associated configuration file because IIS 7.0 includes URLScan. The configuration for the feature in IIS 7.0 is located in the "Request Filtering" section in the applicationHost.config file.
• Add c:\windows\system32\inetsrv to the path. Microsoft.com Operations added the c:\windows\system32\inetsrv folder to the path environment variable so that it could run the Appcmd.exe tool without having to specify the path.
• Remove changes to TCP/IP registry settings. The TCP/IP protocol suite in Windows Server 2008 supports the configuration of more TCP/IP protocol settings without making direct changes to the registry than in Windows Server 2003. Specifically, Microsoft.com Operations removed the TCP/IP SYN (synchronization sequence number) protection key setting TCPMaxHalfOpen because it moved the server farm behind network devices that have built-in SYN protection algorithms. The existing scripts made changes to the registry to configure these settings in Windows Server 2003. Microsoft.com Operations removed the sections of the deployment scripts that made these registry settings.

Validating Non-Microsoft Hardware and Software Products

During the tests performed in the test environment and in the limited pilot deployment in the production environment, Microsoft.com Operations discovered that some hardware and software products were not compatible with Windows Server 2008 and IIS 7.0, specifically backup software and the image deployment products being used for Windows Server 2003 and IIS 6.0.

Prior to the release of a new operating system, these types of issues are common. Microsoft works with hardware and software vendors to resolve these issues so that their products are compatible when the new operating system is released.

In all instances, Microsoft.com Operations was able to do one of the following to resolve the compatibility issues:

• Use a Microsoft product that did not previously exist. This was the solution used to resolve the incompatibility with both the backup and image deployment technologies. Microsoft.com Operations used ImageX to deploy Windows Server 2008 instead of the previous image deployment software. Microsoft.com Operations used Microsoft System Center Data Protection Manager 2007 as its backup solution.
• Upgrade device drivers. Microsoft.com Operations had difficulties with network adapter drivers on some of the computers in the production environment so they worked with the network adapter vendors to resolve the compatibility issues.

Testing the Migration Process in a Lab Environment

Microsoft.com Operations performed the first level of testing in a lab environment. The purpose of these tests was to perform component-level functional testing of features and proof-of-concept testing. Microsoft.com Operations used the lab environment during the early stages of the build process. Almost all of the testing now occurs in a limited pilot deployment in the Microsoft.com production environment.

Testing the Migration Process in Pilot Deployments

After Windows Server 2008, IIS 7.0, the Web applications, installation software, and administration scripts worked properly in the test environment, Microsoft.com Operations started testing in a limited pilot deployment in the production environment. These pilot deployments eventually became full production deployments if the solution worked properly in pilot deployments. Otherwise, if a failure occurred, Microsoft.com Operations analyzed the failures, resolved possible issues, and repeated the tests until successful.

Microsoft.com Operations performed the following sequence to conduct pilot deployments:

1. Remove a production Web server from active rotation. Microsoft.com Operations removed a server from active rotation of incoming traffic by configuring the hardware load balancer for the site. This prevented the server from receiving incoming requests from the Internet.
2. Deploy Windows Server 2008 and IIS 7.0. Microsoft.com Operations manually deployed the latest Windows Server 2008 build to the server removed from active rotation. Next, it installed IIS by using the ServerManagerCmd.exe command-line tool. All the servers were located in remote data centers, so Microsoft.com Operations used remote management hardware to remotely access and image its servers. These steps helped ensure that the process for deploying Windows Server 2008 produced the proper platform for deploying the Web application. In this step, Microsoft.com Operations used ServerManagerCmd.exe to deploy IIS 7.0 and the IIS 7.0 role services.
3. Deploy and configure IIS and custom Web applications. Microsoft.com Operations deployed the Microsoft.com custom Web applications and IIS settings to the server by using the build script that it tested in the test environment. This step helped ensure that the installation software properly installed and configured the applications on IIS 7.0 and Windows Server 2008. In this step, Microsoft.com Operations used the Microsoft.com build script.
4. Verify proper operation of the Web application. After it deployed and configured IIS 7.0 and the Web applications, Microsoft.com Operations performed tests to verify that the site was functioning properly. In this step, Microsoft.com Operations used:
   • TinyGet.exe to perform functional-level testing of the build.
   • Web Capacity Analysis Tool (WCAT) to perform stress testing on the build.
   • Windows PowerShell scripts as a wrapper to automate TinyGet.exe and WCAT.
   • Log Parser to analyze the IIS 7.0 logs to ensure that the server was functioning correctly after running the TinyGet.exe and WCAT tests.
   • ImageX to create a reference image of the server that would be used later in the deployment process.

   For more information about these and other tools used in the testing and deployment processes, refer to Appendix A at the end of this paper.

5. Add the server into active rotation. After the server successfully passed the tests, the server was added back into active rotation.
6. Verify proper operation with production traffic. Microsoft.com Operations closely monitored the server put into active rotation to ensure proper operation. Microsoft.com Operations used the same monitoring technologies and process used in the production environment.
7. Deploy to additional computers in farm. When the build was successful, Microsoft.com Operations performed incremental deployments to additional computers in the Web server farm, carefully monitoring for any problems that might have arisen. This process continued until all computers in the Web server farm were migrated. Microsoft.com Operations used ImageX to deploy the reference image to new servers. The image was created earlier in step 4.

Deploying in the Production Environment

After completing a successful pilot deployment, Microsoft.com Operations completed the deployment of the new build to the rest of the Web servers in the production environment. Microsoft.com Operations followed the same deployment steps that it performed earlier in the pilot deployments.

In addition to the steps performed in the pilot deployments, Microsoft.com Operations performed the following tasks in sequence to deploy a new build in the production environment:

1. Communicate and coordinate with application owners and content providers. A vital part in the rollout to the production environment was having the application owners and content provider validate the functionality of their content. The application owners and content providers had a limited window of time to validate the functionality of their applications prior to deploying their content to the production environment. If the application owners or content providers identified a breaking change, the operations team worked closely with the application owners, content providers, and product teams to perform root cause analysis and resolve the problem.
2. Validate applications and content. To validate all applications and content, the operations team created a preproduction environment that mirrored the production environment but was not Internet facing. In the preproduction environment, the application owners and content providers validated the functionality of their content and applications prior to a wider deployment into the production environment.
3. Deploy in production environment. After the application owners and content providers validated the applications and content, Microsoft.com Operations created a final image of the current configuration. Then, Microsoft.com Operations removed a server from active rotation, upgraded the server to the final image, and then returned the server to active rotation. The team continued this process until it upgraded all servers.
4. Verify proper operation with production traffic. As in the pilot deployment, Microsoft.com Operations closely monitored the health of the servers taking live traffic. Microsoft.com Operations used Microsoft System Center Operations Manager 2007 and other tools for this monitoring so that the team could quickly resolve any issues that arose. This step occurred throughout the entire deployment process in the production environment.
5. Collaborate with the IIS product team. A critical part of successful deployment was working with the IIS product team. The Microsoft.com Operations and IIS product teams met frequently during the deployment to address any outstanding issues with IIS and to provide feedback on the readiness of the component's features for deployment in large-scale environments. This step occurred throughout the entire deployment process in the production environment.

Operating Microsoft.com After Migration

Compared to prior versions of IIS, IIS 7.0 has significant improvement in overall manageability and administration. Configuration control and diagnostic improvements help reduce the effort to manage Microsoft.com on a day-to-day basis. IIS 7.0 also supports an improved delegation of administration model, which gives application and content owners more control to customize the user experience with their Web applications and content.

Improving Configuration Control

The IIS configuration settings are stored in the applicationHost.config file, which is a significant improvement over the metabase in prior version of IIS. Prior to IIS 7.0, Microsoft.com Operations had to run scripts on each remote Web server on the site to update IIS-related configuration settings. The potential always existed for configuration issues if the scripts failed. Configuration-related problems were also introduced if an administrator made a configuration change for troubleshooting and forgot to reverse the changes when the troubleshooting was complete.

Microsoft.com Operations updates a single version of the applicationHost.config file and then copies that version across all servers so that all servers are configured the same. This single version of the configuration file has helped reduce the time to troubleshoot site issues by dramatically reducing configuration-related issues.

Microsoft.com Operations saves the existing applicationHost.config file prior to any configuration change. This helps ensure that the team can quickly restore the previous configuration if an issue arises on the site after the deployment of an updated applicationHost.config file.

The documentation of configuration settings is necessary for any enterprise Web site. Because the applicationHost.config file is in XML format, it can be easily read and the site configuration settings are clearly documented.

Finally, the applicationHost.config file provides Microsoft.com Operations with improved disaster recovery. If a catastrophic failure occurs on the site, Microsoft.com Operations has an easy and repeatable method for quickly recovering the site configuration settings.

Delegating Configuration to Application and Content Owners

One of the features that will dramatically reduce the operational effort for the operations engineers managing the Microsoft.com site is the delegation of many Web sites and application configuration settings to the application and content owners.

Microsoft.com Operations has enabled the delegation of specific configuration settings, such as HttpErrors, HttpRedirect, and defaultDocument, on the Microsoft.com site. The site or application owners can override the delegated configuration settings in the web.config files of their applications. This helps reduce the administrative burden on the site administrators and enables the application owners to customize the user experience. Microsoft.com Operations performed careful planning to decide which settings to delegate on its site.

For the complete list of settings that can be delegated, refer to the IIS.net site (http://www.iis.net). For more information about how to delegate configuration, refer to "How to Use Configuration Delegation in IIS 7.0" at http://learn.iis.net/page.aspx/157/how-to-use-configuration-delegation-in-iis-7/.

Improving Monitoring and Diagnostic Capabilities

Microsoft.com Operations is dedicated to ensuring the highest possible uptime and performance for all the sites that it manages. Closely monitoring the Web applications and the Web servers on Microsoft.com is an important part of the team's proactive efforts to reduce outages and performance problems on that site. Microsoft.com Operations monitors the health and uptime of sites by using Microsoft products such as Microsoft System Center Configuration Manager 2007 and non-Microsoft products.

One of the biggest monitoring and diagnostic improvements in IIS 7.0 is the Failed Request Tracing feature. The Failed Request Tracing feature enables the operations team to proactively set specific IIS request parameters to be traced. The tracing rules can be configured with parameters such as file type or HTTP status code. The tracing rules can be set globally for all URLs or to a specific URL. Therefore, an administrator can configure failure conditions, and after those conditions are met, IIS creates logs that contain the tracing information about the failure condition.

Before the Failed Request Tracing feature, an administrator would act reactively and parse logs created by IIS to diagnose a failure or manually attempt to reproduce the failure condition. Reactively parsing the IIS logs is not very effective because many failure conditions, such as HTTP status 500 errors, occur mainly when the server is under stress.

Microsoft.com Operations also monitors real-time requests by using Appcmd.exe. Appcmd.exe displays a listing of the outstanding requests that a server is serving at that moment. Microsoft.com Operations uses this feature of Appcmd.exe when it is looking for requests that are slow and have large values for time taken (for example, appcmd list request). The time taken value is the length of time that a request takes to be processed.

Publishing Content by Using DFSR

Microsoft.com Operations publishes a subset of the content to the servers that host the Microsoft.com site by using the Distributed File System Replication (DFSR) service. The DFSR service is a new multi-master replication engine that keeps folders synchronized on multiple computers.

Microsoft.com Operations uses DFSR to replicate content to all the computers in the Web farm from a centralized cluster of computers that contain the most current version of content for Microsoft.com. Site content owners update, preview, and validate the content on the centralized staging server. Next, content owners publish their content to the live site. Microsoft.com Operations uses DFSR to replicate and synchronize the content on the Internet-facing Web farm

DFSR includes improvements in performance for replicating content over low-bandwidth connections. Aside from replicating within a data center, DFSR replication occurs between the several Internet data centers that host the Microsoft.com site.

For more information about DFSR, refer to:

• "Distributed File System Replication" at http://msdn2.microsoft.com/en-us/library/bb540031(VS.85).aspx.
• "Overview of the Distributed File System Solution in Microsoft Windows Server 2003 R2" at http://technet2.microsoft.com/windowsserver/en/library/d3afe6ee-3083-4950-a093-8ab748651b761033.mspx?mfr=true.

Planning for Future Improvements

As with any other high-volume Web site, Microsoft.com is constantly undergoing upgrades, new builds of content and code, deployment of service packs, and other changes. Even though running Microsoft.com on IIS 7.0 and Windows Server 2008 is a significant improvement over running the site on previous versions of IIS and Windows Server, Microsoft .com Operations continues to improve uptime, improve performance, and reduce administrative effort.

As a part of its ongoing improvements to Microsoft.com, Microsoft.com Operations is testing and planning to:

• Reduce content deployment effort by using the UNC Content feature. The UNC Content feature in IIS 7.0 enables content to be stored on a shared network folder and then accessed by multiple geographically located computers in a Web farm. Currently, all content on the Microsoft.com site is replicated to each front-end server. Ideally, Microsoft.com Operations wants to reduce the footprint of the front-end Web servers by moving the content to centrally shared file servers and accessing it over Universal Naming Convention (UNC). This would dramatically reduce the number of replication end-points compared to the current content replication implementation.
• Improve performance and disaster recovery by placing the back-end database servers running SQL Server in other geographic locations. Currently, Microsoft.com Operations has the database servers more centrally located. With the TCP/IP stack improvements in Windows Server 2008, moving the back-end SQL Server instances to other geographic locations will enable Microsoft.com Operations to greatly improve end-to-end disaster recovery.
• Improve performance by enabling the output caching feature. The dynamic output caching feature in IIS 7.0 helps to improve performance on Web servers, sites, or applications. When a user requests a Web page, IIS processes the request and returns a page to the client browser. If output caching is enabled, a copy of that processed Web page is stored in memory on the Web server and returned to client browsers in subsequent requests for that same resource. Output caching thus eliminates the requirement to reprocess the page every time that it is requested. This is helpful when content relies on an external program for processing, such as a Common Gateway Interface (CGI) program, or includes data from an external source, such as a remote share or a database.

Best Practices

Microsoft.com Operations has gained practical, real-world experience with designing, deploying, administering, and operating Microsoft.com by using IIS 7.0, Windows Server 2008, and SQL Server 2005. Because of this experience, Microsoft.com Operations recommends the following best practices in the areas of deployment and architecture:

• Deploy applicationHost.config to multiple computers in a Web farm by using xcopy. One of the advantages to the improved configuration model is that IIS configuration settings can be deployed to multiple computers in a Web farm via xcopy. In previous versions of IIS, administrators configured IIS by running scripts that configured multiple places, such as the registry, IIS metabase, and web.config file. Because all the configuration settings are now in the applicationHost.config file, copying the one file copies all the configuration settings.

  Note: Deploying the applicationHost.config file while a server is in active rotation can affect Web server performance. As a best practice, an organization should deploy the applicationHost.config file during non-peak hours of operation.

• Compress traffic between Web farms and clients by using dynamic compression. Dynamic compression reduces the amount of network traffic between the computers in the Web farm and client computers. However, compression increases processor utilization on the Web farm. An organization should analyze the network traffic and the processor utilization of the Web farm to determine which resource is most constrained. If network traffic is most constrained, the organization should enable dynamic compression.
• Reduce the migration process by initially running applications in application pools configured for classic mode. When migrating sites from a previous version of IIS, an organization can help reduce the time required to migrate existing applications to IIS 7.0 by running the Web applications in application pools configured for classic mode. Running applications in classic mode enables an organization to run applications with minimal effort.
• Run existing applications in integrated mode. Running applications in application pools configured to run in integrated mode helps ensure that applications run at optimal performance and can access the latest features available in IIS 7.0. An organization should migrate existing applications by using the Appcmd.exe command-line tool with the/migrate option.
• Install the IIS 6.0 Management Compatibility components. Organizations typically run existing configuration and management scripts by installing the IIS 6.0 Management Compatibility components. However, an organization should use this method only as a short-term solution for migration. A long-term solution is to rewrite existing code to use the new Windows PowerShell provider, WMI provider, IIS administration tools, and APIs.
• Review online forums for most current information. The forums, blogs, communities, and Download Center on the Microsoft.com or IIS.net site provide the latest information about IIS 7.0 installation, migration, development, and operations. An organization should review these online resources frequently to find the latest information about these topics.

Conclusion

Deploying any software to a high-volume Internet Web presence requires careful planning, extensive testing, proven processes, and the right tools. For the migration to IIS 7.0 and Windows Server 2008, Microsoft.com Operations had all those essential components. The migration occurred with minimal outages and minimal development effort. In fact, the migration effort was only about as complicated as deploying a service pack.

After migration, Microsoft.com Operations was able to take advantage of the improved administration model, simplified configuration, improved security, improved monitoring, and improved identification of potential problems. In the future, Microsoft.com intends to make use of centralized content storage and output cache features.

By migrating high-volume Web sites to IIS 7.0 and Windows Server 2008, an organization can reduce the effort to administer and support solutions. In addition, an organization can take advantage of the security and performance features in IIS 7.0 and Windows Server 2008. It can gain all of these benefits with minimal risk and effort in the migration process.

For More Information

For more information about operations for Microsoft.com, refer to the Microsoft.com Operations blog at:

http://blogs.technet.com/mscom/

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 2008 Microsoft Corporation. All rights reserved.

Microsoft, Active Directory, MSDN, SQL Server, Windows, Windows PowerShell, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

All other trademarks are property of their respective owners.

Appendix A: Tools Used in Testing and Deployment Processes

Before deploying Windows Server 2008 and IIS 7.0 in the test and production environments, Microsoft.com Operations identified tools that could help automate the deployment processes and the test processes. Microsoft.com Operations used the deployment automation tools to install Windows Server 2008, IIS 7.0, and the Web applications. Microsoft.com Operations used the test automation tools to help validate the proper operation of the Web applications in the test environment and in the pilot deployments in the production environment.

The tools that Microsoft.com Operations used in its testing and deployment processes included:

• ServerManagerCmd.exe
• TinyGet.exe
• Web Capacity Analysis Tool
• Windows PowerShell
• Log Parser
• Appcmd.exe
• PsExec

ServerManagerCmd.exe

Microsoft.com Operations used ServerManagerCmd.exe during the deployment process to install the Web Services (IIS) server role and features. ServerManagerCmd.exe ships with all editions of Windows Server 2008.

The following is an example of how to use the ServerManagerCmd.exe command-line tool to install IIS 7.0 and the appropriate IIS 7.0 features.

        ServerManagerCmd.exe -I Web-Server Web-Common-Http Web-Http-Redirect Web-Asp-Net
        Web-Net-Ext Web-ASP Web-ISAPI-Ext Web-ISAPI-Filter Web-Http-Logging Web-Log-Libraries
        Web-Request-Monitor Web-Http-Tracing Web-Filtering Web-Stat-Compression Web-Dyn-Compression
        Web-Mgmt-Console Web-Scripting-Tools Web-Mgmt-Service Web-Mgmt-Compat WAS

For more information about ServerManagerCmd.exe, refer to:

• "Overview of Server Manager Commands " at http://technet2.microsoft.com/WindowsVista/en/library/10c24c6a-44e0-4226-89f5-c2a0cec7d0111033.mspx .
• "Server Manager Technical Overview Appendix" at http://technet2.microsoft.com/windowsserver2008/en/library/e7edce1d-442c-4ec3-b324-c748e4f937551033.mspx?mfr=true.

TinyGet.exe

TinyGet.exe is a command-line tool that can be downloaded as a part of the IIS 6.0 Resource Kit. Microsoft.com used it to troubleshoot, test, and log HTTP connections between server and client computers. TinyGet.exe enabled Microsoft.com Operations to customize each test request by configuring many different factors, including the authentication method, HTTP version, and output format. TinyGet.exe also enabled Microsoft.com to use scripts that specify looping and multithreading.

Microsoft.com Operations used TinyGet.exe to replay its IIS logs in the test and pilot deployment environments to verify proper functionally. The team obtained the IIS logs from existing servers in active rotation. They reviewed the results of the TinyGet.exe tests to identify any breaking changes to code, content, or other parts of the Web applications. If the build being tested passed the TinyGet.exe functional testing, Microsoft.com Operations performed stress testing on the build by using WCAT. Microsoft.com Operations automated the TinyGet.exe-based tests by using a Windows PowerShell script wrapper.

The following is an example of how to use the TinyGet command-line tool to replay an IIS log on the target server.

 
        C:\tinyget start SourceServerName IISLogFileName.log TargetServerName
    
        Statistics:
    
        -----------
    
        Elements processed: 843019
    
        Elements output: 1000
    
        Execution time: 8.97 seconds
    
        Script name: IISLogFileName.scr
    
        Testcase number: 39 - Explain: (null)
    
        URI: /latam/athome/security/videos/undefined?404;http://www.microsoft.com:80/latam/athome/security
    
        ERROR: 0x4b8 : returned status code (404) does not match expected one (200) received
        error: (404 Not Found)
    
        Testcase number: 146 - Explain: (null)
    
        URI: /poland/windows/ie/images/newsletter_120x240.jpg, SSL: Nonsecure, CliCert:
    
        ERROR: 0x4b8 : returned status code (301) does not match expected one (404) received
        error: (301 Moved Permanently)

For more information about TinyGet.exe, refer to "TinyGet (TinyGet.exe)" in "IIS 6.0 Resource Kit Tools (IIS 6.0)" at http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/993a8a36-5761-448f-889e-9ae58d072c09.mspx?mfr=true.

Web Capacity Analysis Tool

WCAT is a tool for capacity testing and planning that can be downloaded as a part of the IIS 6.0 Resource Kit. Microsoft.com Operations used WCAT to test different server and network configurations by using custom-designed content and workload simulations.

WCAT contains two tools, Wcclient.exe and Wcctl.exe, which simulate Web requests to perform stress and performance testing on a Web server. Wcclient.exe is the stress client that creates the simulated traffic. Wcctl.exe initiates and monitors the tests. Microsoft.com Operations automated the WCAT-based tests by using a Windows PowerShell script wrapper.

Microsoft.com Operations ran WCAT after the new build had passed the TinyGet.exe functionality testing, while the computer was still out of active rotation. If the build passed the WCAT stress tests, Microsoft.com Operations placed the computer back into active rotation and allowed limited Internet traffic to the computer.

The following is the syntax and sample output from WCAT.

        C:\wcat>wcctl.exe -t u_ex07071108_HOT.ubr -v 30 -s localhost -c 1 -w 0 -n 0 -u
        7200
    
        Where :
    
        -v is virtual clients (you should use 500)
    
        -u is the run duration
    
        -w warmup
    
        -n cooldown
    
        ********************************************************************
    
        DURATION 7200/7200 secs 2 hours, 30 minutes, 5 seconds
    
        ********************************************************************
    
        Connections = 9395 ( 1/sec)
    
        Disconnects = 9281 ( 1/sec)
    
        Socket Sends = 2934790 ( 407/sec)
    
        Socket Receives = 6355801 ( 882/sec)
    
        Full Handshakes = 0 ( 0/sec)
    
        Reconnect Handshakes = 0 ( 0/sec)
    
        Bytes Sent = 961785206 ( 133581/sec)
    
        Bytes Received = 38640469462 ( 5366731/sec)
    
        Bytes Sent (SSL) = 0 ( 0/sec)
    
        Bytes Received (SSL) = 0 ( 0/sec)
    
        Time To First Byte = 0 ( Minimum)
    
        Time To First Byte = 123890 ( Maximum)
    
        Time To First Byte = 1125 ( Average)
    
        Time To Last Byte = 0 ( Minimum)
    
        Time To Last Byte = 123890 ( Maximum)
    
        Time To Last Byte = 1127 ( Average)
    
        Transactions = 2944186 ( 408/sec)
    
        Normal Requests = 2944186 ( 408/sec)
    
        Secure Requests = 0 ( 0/sec)
    
        Normal Responses = 2935339 ( 407/sec)
    
        Secure Responses = 0 ( 0/sec)
    
        Status 200 = 2917033 ( 405/sec)
    
        Status 301 = 2310 ( 0/sec)
    
        Status 302 = 12372 ( 1/sec)
    
        Status 400 = 661 ( 0/sec)
    
        Status 404 = 2717 ( 0/sec)
    
        Status 500 = 263 ( 0/sec)
    
        Total Errors = 26953 ( 3/sec)
    
        Connect Errors = 0 ( 0/sec)
    
        Send Errors = 0 ( 0/sec)
    
        Receive Errors = 8620 ( 1/sec)
    
        Parsing Errors = 0 ( 0/sec)
    
        Unexpected Status = 18333 ( 2/sec)

The latest version of WCAT is version 6.3 and is available for download on IIS.net as follows:

• WCAT 6.3 (x86) at http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1466
• WCAT 6.3 (x64) at http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1467

For more information about WCAT, refer to "WCAT (Wcclient.exe and Wcctl.exe)" in "IIS 6.0 Resource Kit Tools (IIS 6.0)" at http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/993a8a36-5761-448f-889e-9ae58d072c09.mspx?mfr=true.

Windows PowerShell

IIS 7.0, Exchange Server 2007, System Center Operations Manager 2007, System Center Data Protection Manager 2006, and Microsoft System Center Virtual Machine Manager also use Windows PowerShell to improve administrator control, efficiency, and productivity.

Microsoft.com Operations used Windows PowerShell to automate common deployment, administrative, and operations tasks. Specifically, Microsoft.com Operations automated TinyGet.exe and WCAT-based tests by using a Windows PowerShell script wrapper.

For more information about Windows PowerShell, refer to "Scripting with Windows PowerShell" at http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx.

Log Parser

Log Parser provides universal query access to text-based data such as log files, XML files, and comma-separated values (CSV) files, as well as key data sources on the Windows operating system, such as the event log, the registry, the file system, and Active Directory. The latest version of Log Parser can be downloaded from http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en.

Log Parser contains two components, LogParser.exe and LogParser.dll. LogParser.exe is a command-line version of the tool. LogParser.dll is a set of Component Object Model (COM) objects that can be used in scripts.

Microsoft.com Operations used Log Parser to parse and analyze the Microsoft.com IIS logs for trends in health and performance on the site. The team reviewed this information before adding a new server into active rotation.

The following is an example of how to use the LogParser.exe command-line tool to display the HTTP status codes for a cluster of computers.

        logparser -rtp:-1 file:statuscodes_by_servers_in_cluster.sql?logfilename=u_ex07111409.log
    
        sc-status Srv1 Srv2 Srv3 Srv4 Srv5 Srv6 Srv7 Srv8
    
        --------- ------ ------ ------ ------ ------ ------ ------ ------
    
        200 576313 585078 573956 574218 566388 599807 572982 589815
    
        206 242 238 247 262 236 216 192 246
    
        301 6148 6388 6378 6154 6140 6600 6293 6398
    
        302 166817 168700 166471 165909 163386 172658 166764 170363
    
        304 125493 126640 125908 126204 124139 130669 125855 129026
    
        400 4 3 3 2 2 7 7 3
    
        403 130 106 146 170 148 154 148 95
    
        404 9023 9149 9835 8833 8811 9489 9250 9369
    
        405 0 1 0 0 0 0 0 0
    
        406 2 10 5 11 4 7 8 4
    
        500 1361 342 363 337 332 323 340 351
    
        501 18 22 21 25 31 27 34 30

Appcmd.exe

Windows Server 2008 and IIS 7.0 contain a new command-line tool, Appcmd.exe. Appcmd.exe can be used to manage and administer Web servers, Web sites, and Web applications. The command-line interface simplifies common management Web server tasks for administrators. Appcmd.exe replaces the Adsutil.vbs administration script used in IIS 6.0.

For example, an administrator can use Appcmd.exe to list Web server requests that have been forced to wait for more than 500 milliseconds. The administrator can use this information to troubleshoot applications that are performing poorly. The output of Appcmd.exe can be redirected to other commands for further processing.

Microsoft.com Operations used Appcmd.exe to automate the configuration of IIS 7.0. Microsoft.com Operations automated Appcmd.exe by using a PowerShell script wrapper.

The following is an example of how to list the currently running requests by using Appcmd.exe.

        C:\Windows\System32\inetsrv>appcmd list request
    
        REQUEST "cd00000380021ba4" (url:GET /en/us/default.aspx, time:11594 msec,
        client:10.0.0.1, stage:SendResponse, module:IIS Web Core)
    
        REQUEST "9300008080003951" (url:GET /en/us/default.aspx, time:11125 msec,
        client:10.0.0.1, stage:SendResponse, module:IIS Web Core)
    
        REQUEST "fc0000828001a096" (url:GET /en/us/default.aspx, time:10516 msec,
        client:10.0.0.1, stage:SendResponse, module:IIS Web Core)
    

The following is an example of how to list the current application pools on a computer running IIS 7.0 by using Appcmd.exe.

        C:\Windows\System32\inetsrv>appcmd list apppool
    
        APPPOOL "Default" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "AppPool1" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "AppPool2" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "AppPool3" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "AppPool4" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "AppPool5" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
    
        APPPOOL "Classic" (MgdVersion:v2.0,MgdMode:Classic,state:Started)

For more information about Appcmd.exe, refer to "IIS 7.0: Appcmd.exe" at http://technet2.microsoft.com/WindowsServer2008/en/library/ec52c53b-6aff-4d76-995e-3d222588bf321033.mspx.

PsExec

PsExec is part of a growing kit of Sysinternals command-line tools (named PsTools) that aid in the administration of local and remote Windows operating systems. Sysinternals is a set of tools that help manage, troubleshoot, and diagnose Windows-based systems and applications. Tools like Telnet and remote control programs like Remote Desktop Connection in Windows operating systems enable administrators to run programs on remote systems, but they can be complex to install and are not easy to automate.

PsExec is a lightweight Telnet replacement that enables administrators to run processes on other systems, complete with full interactivity for console applications, without having to manually install client software. The most powerful uses of PsExec include opening interactive command prompts on remote systems and remote-enabling tools like IPconfig that otherwise do not have the ability to show information about remote systems.

Microsoft.com Operations used PsExec to run commands on the Web servers in the Web farms that host Microsoft.com. For more information about PsExec, refer to "PsExec v1.94" at http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx.