Export (0) Print
Expand All

Microsoft IT Starts Migration of Microsoft.com to Windows Azure Platform

Article

Published: September 2010

The Video Showcase site on Microsoft.com hosts thousands of videos and other rich media content that hundreds of thousands of customers worldwide view every day. Learn how Microsoft Information Technology (MSIT) quickly built and deployed the Social eXperience Platform (SXP) on the Windows Azure™ technology platform to enable social media capabilities across Microsoft.com.

Download

Intended Audience

Products & Technologies

Download Article, 480 KB, Microsoft Word file

Technical decision makers and business decision makers

  • Windows Azure
  • SQL Azure
  • Microsoft .NET
  • Microsoft Silverlight
  • Microsoft Visual Studio
  • Microsoft System Center Operations Manager
  • Social eXperience Platform

Introduction

Microsoft has adopted a "we’re all in" strategy for the cloud. As part of this strategy, MSIT is migrating Microsoft.com to the cloud via Windows Azure. SXP is one of the first technologies that MSIT has built and deployed in the cloud by using Windows Azure. This article focuses on the decisions that the SXP team made when creating the Windows Azure-based service. The article discusses the many advantages that Windows Azure provides and also discusses lessons learned.

The Video Showcase Site on Microsoft.com

The Video Showcase site on Microsoft.com hosts more than 8000 videos that assist Microsoft customers in their search for information about products and services. The marketing agencies that Microsoft works with use the site as a video repository. The typical video is four to five minutes long. Customers view and learn from the video content and can comment on, rate, and share videos with their friends and colleagues on other social media channels and sites. The goal is to match customer intent with Video Showcase content and to create a sense of community and interest in Microsoft products.

The user experience for the site was developed with Microsoft® Silverlight® technology. The Video Showcase site previously used a third-party on-premises solution to manage comments and ratings, to filter out profanity and spam, and to do site moderation. This solution used blogs and blog posts to enable comments. Although the solution worked, it was hard to understand and explain. Maintenance costs were high and there were also issues related to upgrades, scalability, performance, and availability. Also, the business intelligence (BI) and information related to ratings and comments were either difficult to access or not available at all. For these reasons, MSIT developed SXP, which is essentially a platform as a service (PaaS).

To learn more about the Video Showcase site, visit http://www.microsoft.com/showcase/en/US/default.aspx.

The Social eXperience Platform

SXP is a multi-tenant Web service that enables social media capabilities across Microsoft.com. SXP’s first tenant is the Video Showcase site. Customers can use the SXP Windows Azure-based service to manage comments and ratings, and to filter out profanity and spam. SXP provides the ability to comment on or rate anything, including Web pages, news stories, videos, blog posts, and so on. As long as a unique ID can be assigned, users can comment on or rate the item. The SXP team manages comments and filters via the SXP Moderation Tool (built with Silverlight), which is also available via Windows Azure.

SXP also provides a Really Simple Syndication (RSS) feed aggregation, grooming, and publication service to all Microsoft.com sites. This service enables marketing site managers to easily manage and subscribe to relevant content across both desktop computers and mobile devices.

The SXP service is a back-end service for comments and ratings; it is not the front-end Web server. The service does not host the videos (they are hosted on MSN® Video Web Services) and does not host the HTML that users see on the screen.

SXP Architecture

The SXP solution takes advantage of the best of both on-premises and cloud-based technologies. SXP runs on Windows Azure and the SQL Azure™ Database, but the Video Showcase Web servers are standard Microsoft.com servers running in Microsoft’s internal data center. This combination of technologies enabled the SXP team to quickly create a solution with a high level of interoperability that adheres to the security and governance requirements that are typical of a large enterprise.

Figure 1 shows the SXP architecture.

Ff973966.image001(en-us,TechNet.10).jpg
Figure 1. SXP architecture

SXP Web Services. The bulk of the work occurs in the SXP Web Services—a Representational State Transfer Service (REST) running on the Microsoft Visual Studio® 2010 development system and Microsoft .NET 3.5. SXP Web Services is a single Web role that runs on three medium instances. The SXP team chose three instances to provide the high availability and scalability necessary for an enterprise Web site the size of Microsoft.com. The service currently runs at about 5 percent utilization, so it has a lot more capacity than it needs. As the service brings on more partners, the team expects utilization to increase. Windows Azure provides nearly unlimited scalability for future growth.

The service is multi-tenant, allowing any sub site on Microsoft.com to use the social capabilities of SXP. The SXP team accomplishes multi-tenancy by creating a separate SQL Azure database for each tenant to store comments and ratings. The team chose this approach to take advantage of the separation that multiple databases provide. Using multiple databases can also help with availability—the team has never had two databases become unavailable at the same time. Multiple databases also offer a small degree of customization across tenants.

In contrast to having one database per tenant, log and configuration data are shared per instance.

SXP Monitor. The SXP Monitor is a small application that the team built. The SXP Monitor runs in the same data center as the Web service, so there is no network in the way. The SXP Monitor runs on a single Web role that runs a single instance.

The SXP Monitor:

  • Pings the Web service every 15 seconds to determine whether the service is up. Because the SXP Monitor is in the same data center as the Web service, the service should be available 100 percent of the time. If the SXP Monitor indicates any downtime, the team knows that a problem exists in the data center.
  • Trims the Windows Azure Diagnostics storage tables. Windows Azure sends a lot of diagnostics information, but does not clean up that information when it is no longer needed. This would cause the service to slow down if it were querying against many gigabytes of data. Querying the Windows Azure tables is not like querying Microsoft SQL Server® tables because there are no secondary indexes with Windows Azure tables. The SXP Monitor trims the data and keeps just 30 days at all times. The Windows Azure System Center Management Pack will have the same data-trimming functions built in, so the SXP team will retire the data-trimming functions of the SXP Monitor when it migrates to the management pack.
  • Provides quick access to the Windows Azure Diagnostics storage tables. The SXP Monitor pulls the data back and sends it as a Microsoft Excel® tab-delimited file. This is a quick way to pull back all of the data without having to page through 1,000 rows at a time, which can take a fair amount of time through remote access.
  • Pulls metadata for videos from the MSN Video Web Services. After a specified amount of time, the SXP Monitor pulls new data and caches it into the SXP asset tables.

Windows Azure Storage. The service uses Windows Azure Storage just for diagnostics and log information.

SCOM Connector .The SCOM Connector provides alerting and notification via Microsoft System Center Operations Manager. The service currently uses custom code to integrate the SCOM Connector, but the SXP team is also running the release candidate version of the Windows Azure System Center Management Pack in side-by-side deployment and plans to migrate to the management pack as soon as it is available. The service gets about 200,000 log entries per day.

Keynote . In addition to the monitoring done through the SXP Monitor, the team monitors through the third-party Keynote product. Keynote pings the Web service from 23 different locales around the world, providing uptime information from an end-user perspective.

Moderation Tool. The Moderation Tool manages comments and filters. The Moderation Tool is authenticated by Active Directory® Federation Services (AD FS), which allows extending Active Directory Domain Services (AD DS) authentication into the cloud.

Results

The service went live in April 2010. After 120 days of operation, the SXP team compared results with the previous third-party solution for the same calendar quarter. Table 1 summarizes the before and after statistics.

Table 1. Comparison of third-party solution to SXP solution over a 120-day period

Focus area Before After

Availability

99.1 percent

99.98 percent

IT support costs

$15,000 per month

$1,050 per month (standard customer-facing pricing)

Planned downtime

2 to 4 hours

0 hours

Hardware provisioning time

2 to 3 weeks for virtual machines; 5 to 6 weeks for physical servers

30 minutes for server environment provisioning

Availability. The table shows that migrating from the third-party product to SXP and Windows Azure dramatically increased service availability. Availability is measured at the transaction level. On most days, the service is available 100 percent of the time. In the month of July 2010, the service was available 99.999 percent of the time. In fact, Windows Azure running in the South Central data center has a similar availability profile to the Microsoft.com data center in Eastern Washington. This means that for a fraction of the cost, developers can comfortably deploy a Windows Azure application and get about the same availability worldwide as the Microsoft.com data center. The SXP team expects that the availability gains that Windows Azure makes possible will reduce downtime by more than 48 hours over a one-year period.

IT Support Costs. Windows Azure makes possible the idea of "utility computing." With utility computing, businesses can take advantage of toll-based hardware. This is in contrast to on-premises (purchased) hardware or hosted (leased) hardware. For the SXP service, utility computing results in monthly savings of almost $14,000 compared to the previous solution. For the month of July 2010, the service delivered 2.4 million transactions at a cost of $0.0000965 per transaction, amounting to a total cost of $1,050 for the entire month (standard Windows Azure pricing). It is important to keep in mind, however, that the SXP service is running at only 5 percent utilization. For the $1,050 that the team pays per month for the production Windows Azure data center, SXP could support 10 times the current load.

Planned Downtime. With Windows Azure, service releases are literally push-button activities with zero planned downtime. The SXP team uses the Virtual IP (VIP) Swap feature on the Windows Azure Portal to promote its staging environment to a production environment. The team has upgraded the service four times by using this feature and has not missed a single transaction. Having such smooth upgrades saves money because the team does not have to duplicate its entire environment.

Hardware Provisioning Time. By using Windows Azure, MSIT has reduced the time it takes to order, set up, and repurpose hardware from weeks to minutes. With the new solution, provisioning the servers takes less than 30 minutes because there is no need to get budget approval, order hardware, and rack, stack, and configure the servers.

Key Performance Indicators

The SXP team wanted to be able to determine the value of the SXP service on the Video Showcase site. The team wanted to know how many customers it was reaching, the engagement level, and the satisfaction level. Table 2 shows the business key performance indicators (KPIs) for the period from product launch through May 2010.

Table 2. KPIs from product launch through May 2010

Reach Engagement Satisfaction Value (of SXP)

22 million unique visitors (up 11 percent in May)

1.4 million videos viewed (up 3 percent in May)

133 net satisfaction (NSAT) rating (up 4 percent from fiscal year 2009)

3,198 videos with ratings (up 17 percent in May)

51 million page views (up 11 percent in May)

4:49 minutes per visit (flat in May)

 

1,305 videos with comments (up 166 percent in May)

The table shows that number of video views is not large, but the time spent on each video is significant, because customers are being engaged for that amount of time. The SXP team measures NSAT by doing surveys after the fact. The SXP platform enables the team to pull all of this BI data out in real time and examine the data in Excel. With the previous solution, the team did not have access to all of this data. This whole set of BI data can be replicated as the SXP team migrates the service to other sites.

Other Benefits of Developing for Windows Azure

The SXP team also noted the following additional advantages provided by Windows Azure.

Robust and Familiar Development Tools

Windows Azure supports open standards and makes use of familiar tools like Visual Studio and the System Center products. This enabled the SXP team to quickly design, develop, and deploy a highly interoperable REST service that has the performance and on-demand scale that Microsoft.com requires.

Although there are some important differences to keep in mind when designing, developing, and deploying with Windows Azure (particularly if the project requires porting a large code base), to a developer, Windows Azure is basically just Internet Information Services (IIS) plus SQL Server. Because Windows Azure is so prescriptive, many of the architectural decisions have already been made. This may not be appropriate for every software solution, but it worked very well for the SXP team.

Consistent Environment

Because Windows Azure packages and deploys the entire solution in a consistent environment, it prevents time-consuming bugs introduced by configuration errors and differences. As the team moved from development to testing, to pre-production, and then to production, and as it brought in new personnel, it did not experience some of the normal configuration issues associated with Web development.

For previous projects, the testing team spent a lot of time resolving configuration-related bugs. With the SXP project, every deployment that passed the smoke test worked. There is a trade-off for this consistency, because developers cannot just update the Web.config file when re-deploying software. For the SXP team, this tradeoff was worthwhile, but it is an important consideration to keep in mind.

Performance and Scalability

The solution delivers virtually unlimited scalability and significantly increased availability at a lower cost when compared to the on-premises solution. The SXP team caches the configuration data, but for everything else, the team goes back to the database. As the load increases, the team sees great response times. Even when the service experiences three times the normal traffic load, the average response time is still well within the team’s service level agreements (SLAs), and the team has not seen any hung requests.

SXP team members reside in Austin, Redmond, and Hyderabad, and the data center is in the United States. Performance is excellent in Austin and Redmond and is good in Hyderabad. In future releases, the team will take advantage of the Windows Azure data centers in Asia, which will improve performance there.

Creation of Unique Environments

The SXP team found that it is very fast, easy, and inexpensive to create unique environments (development, test, integration, pre-production) by using Windows Azure. When the team is finished with an environment, it can let it go. When the team wants the environment back again, it can do so very quickly. This reduces the cycle time for ramping up and ramping down.

The SXP team also found that Windows Azure makes it easy to create multiple proof-of-concept environments that greatly improve user feedback. The team created several demos for business partners. The team received much better feedback from the demos than from the functional specifications.

Global Development

For teams that are spread throughout the globe, traditional solutions require remote access from one location or another. Windows Azure lends itself to global development. A team can deploy an environment in its own area. For the SXP project, the development team resides in the United States and India. The testing team resides in India.

Deployment Automation and Testing

Windows Azure provides an application programming interface (API) and scripts that fully automate deployment, which reduces human error during the deployment process. Due to the standardization on Windows Azure, the SXP team was able to test its production deployment processes and scripts throughout the software development life cycle. This greatly reduced the last-minute issues often encountered in production deployment.

Lessons Learned

Writing code for Windows Azure is very familiar for developers accustomed to working with Visual Studio and Microsoft .NET, but there are also some important differences to keep in mind. The SXP team noted the following lessons learned:

  • Pick the right application. The primary key to success in developing for Windows Azure is to pick the right application. Windows Azure is great for building a Web application or a compute-intensive application, but it is not yet a general-purpose application development platform. The SXP team started with a lower-risk customer-facing project to validate that everything worked as planned.
  • Prepare for the impact on operations. Operations is the biggest change when developing for Windows Azure. It is critical to understand the operations impact, get the operations team on board early, and design for operations. Because the SXP team knew that it faced an unknown operations environment, the team heavily instrumented its code so that it would know exactly how many transactions succeeded or failed. The team took advantage of the out-of-the-box Windows Azure Diagnostics to do the bulk of the work but also wrote some small custom tools.
  • Prepare early for security and integration. Security and integration are very important considerations. AD FS is a great security solution for authentication. Integration is challenging if there is data inside and outside the firewall. For projects that require integration, it is important to make sure that those problems can be solved before development starts.
  • Build in SQL Azure retries. SQL Azure moves databases to balance load. When a database is moved, the connection pool becomes invalid. If the service does not retry the SQL connection, the connect request fails, causing errors. So it is critical to build in SQL Azure retries. For examples of code that retries connections, see the blog entries at http://blogs.msdn.com/b/bartr/.
  • Conduct performance testing. Running performance and stress tests with Visual Studio and Windows Azure can be challenging. The problem is the wide area network (WAN) link between the client and the server. The results can become skewed, and the network can quickly become a bottleneck. The potential to accumulate bandwidth costs also exists. The SXP team solved this problem by creating a simple application that it deployed in Windows Azure in the same data center, in order to test heavily on the same network. This testing provided feedback on real performance data.

The cloud and Windows Azure are evolving quickly. New scripts and tools are available for developers every day. To stay abreast of changes and find more information, visit the Windows Azure Portal at http://www.microsoft.com/windowsazure/ and the Windows Azure Community site at http://www.microsoft.com/windowsazure/community/.

Conclusion

The SXP team at Microsoft.com created a multi-tenant Windows Azure-based Web service that enables social media capabilities across Microsoft.com. The Video Showcase site at Microsoft.com is the first tenant for the SXP service. Customers can use the SXP service to manage comments and ratings and to filter out profanity and spam. Customers can also use the service as an RSS feed aggregation, grooming, and publication service. The SXP team used familiar development tools to quickly create a solution based on Windows Azure that has extremely high availability, scalability, and performance. The new solution also offers significant cost savings for MSIT.

For More Information

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Order Centre at (800) 933-4750. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information via the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Active Directory, Excel, MSN, Silverlight, SQL Azure, SQL Server, Visual Studio, and Windows Azure are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Was this page helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft