Migrating the Microsoft Partner Network to Windows Azure
Technical Case Study
Published: June 2012
The following content may no longer reflect Microsoft’s current position or infrastructure. This content should be viewed as reference documentation only, to inform IT business decisions within your own company or organization.
The Microsoft Partner Network (MPN) is a community that helps Microsoft partners reach their full potential. Microsoft Information Technology (MSIT) currently is re-architecting and migrating the Partner Velocity Platform, which represents the core of MPN functionality, to the Windows Azure Platform, to increase partner satisfaction, gain business agility, provide better service-level agreements with Microsoft businesses, and reduce operational costs by more than twenty percent over the next three years.
Technical Case Study, 573 KB, Microsoft Word Document
|Solution||Benefits||Products & Technologies|
Microsoft IT wanted to address issues with the Partner Velocity Platform (PVP), which provided the core functionality for the Microsoft Partner Network. The isolated nature of PVP applications was limiting the platform's performance in several areas and impacting the user experience.
The team began migration of PVP to PVP.Next, a solution based on Windows Azure that is designed to replace PVP.
The hybrid architecture created by the PVP.Next team enabled an ongoing migration process that does not impact the user experience negatively. Additionally, it continues to provide an improved user experience and resolve issues associated with the preexisting PVP solution.
The Microsoft® Partner Network (MPN) portal on Microsoft.com is the primary resource for Microsoft partners to establish, manage, and grow their partnership with Microsoft. The MPN portal provides resources and a community that enable partners to strengthen their technical capabilities, gain expertise to help serve customers better, and provide innovative solutions for their customers.
The Partner Velocity Platform
The Partner Velocity Platform (PVP) is a suite of applications that provide functionality and support for the MPN. While PVP does not include all MPN functionality, it is the core functional component of the MPN Partner Portal. As such, it provides the background services and supporting components on which the MPN operates. PVP is an aggregate of different applications that have been amalgamated into the MPN portal, and Microsoft Information Technology (MSIT) supports nearly all PVP components.
The preexisting PVP solution was composed of multiple applications that different teams designed and developed to serve a multitude of purposes within the MPN. Many of these applications were assimilated into PVP. As a result, PVP consisted of a large number of applications, all of which had different interfaces, functional designs, and methods for maintenance and support.
The preexisting PVP solution operated primarily as an on-premises solution, and servers within the Microsoft IT corporate network infrastructure hosted all of the PVP components. PVP applications were designed to access data and services provided through these components, either by using a Windows client or browser-based application.
In this model, users on the Microsoft corporate network are authenticated and granted PVP access through authentication methods that the internal network environment provides. External users, however, must be authenticated by using different methods. These different methods include accessing the on-premises applications through the Microsoft corporate-network firewall, which requires authentication not only to the application, but also to the firewall infrastructure. Additionally, this authentication potentially could be required multiple times.
Preexisting PVP Issues
The preexisting PVP solution was experiencing several issues that were impacting functionality and maintainability negatively on the MPN, including:
There were two factors limiting the performance and scalability of PVP applications in the previous environment. First, PVP applications were distributed among multiple servers within Microsoft data centers, and there were many different maintenance and management models implemented. In the existing configuration, several server resources needed to access a single database (such as the partner database), which acted as the bottleneck. This resulted in decreased application performance during peak usage periods. Additionally, the configuration of the servers made it difficult for MSIT to add resources. They already were using some of the fastest hardware available. As a result, permanent increases in application usage, or the introduction of new applications, caused decreases in peak performance. Overall, the PVP platform was not operating efficiently or cost-effectively.
Complex workflow and poor data quality
The third negative side effect of isolated and individually developed applications was an inefficient workflow process, which resulted in poor data quality. Some applications were not able to share data, which meant that the PVP databases were storing duplicate information. Additionally, data was being orphaned or misrepresented within PVP data structures.
Poor scalability and business agility
PVP, like many user-driven applications, does not experience consistent and predictable usage and traffic. Demand for applications and the resources that these applications used would vary, depending on the application and the business purpose it served. Because of the preexisting infrastructure bottlenecks, PVP was unable to scale properly to meet application demands. Changes to PVP that supported new business requirements were also limited by the existing application structure. New applications and environment changes could take weeks to implement due to the isolation of platform components and hardware limitations.
Fragmented partner experience
From the end-user perspective, the PVP applications did not function as a cohesive platform. Application interfaces were inconsistent. Appearance of user interface (UI) elements, authentication mechanisms, and localization capabilities varied from application to application. Additionally, information was not being shared between all applications, and the user experience suffered.
Lack of social and collaboration capabilities
Because so many of the PVP applications were isolated, maintaining community-oriented functionality was difficult, as was aggregating data across applications. Furthermore, individual applications had no built-in methods to communicate with each other and share information.
The development team assessed the state of the PVP components, and made several high-level design decisions that would drive the next iteration of PVP, suitably named PVP.Next.
One of the most important aspects of the PVP.Next redesign was that the team did not attempt to rearchitect the entire platform at once. Instead, they started with PVP's core functional components, and worked outwards. In the first phase, the team identified three components for migration: Assets, Enrollments, and Requirement. These components represented only a portion of the overall plan for the redesigned PVP.Next infrastructure, and they provided an excellent starting point for the Windows Azure migration project.
For PVP.Next, the team established several design goals:
Create more efficient data structures and workflow
Data structures and their accompanying workflow were the core of PVP. The first, and most critical, design goal was to remedy the limitations of the preexisting design. To provide a platform, the PVP.Next team knew that they would have to realign and redesign the workflow and data structures that supported PVP.Next to support the application-level changes. Data had to be usable and distributable between applications, and the PVP.Next data-management methods across the entire platform needed to be more efficient and effective. The team recognized that fulfilling this goal would be critical to achieving the other goals that they defined.
Increase business value
The PVP.Next team understood that the isolated nature of PVP applications was costly. Therefore, with PVP.Next, they wanted to eliminate the necessity for multiple maintenance and management methods. Additionally, they wanted to unify PVP.Next under a single infrastructure, and provide a more efficient platform.
Increase business agility by reducing development and implementation time
The team knew that it had to respond more quickly to changes in the partner environment, and to requirements for application modification and integration. Therefore, PVP.Next needed to be agile to enable the testing and implementation of changes at the same speed with which the partner experience evolved.
Realign the technology roadmap to correspond with business goals
The preexisting PVP solution was not fulfilling the needs of the business units that were providing the partner experience. Application behavior and integration often was counterintuitive to the desired partner experience. Therefore, the team needed to align the PVP.Next functionality and workflow to enable a more logical integration with business operations and goals.
Develop a single work surface for the entire partner experience
Multiple, disparate applications in the previous PVP version resulted in a multitude of possible user experiences and a general disconnect between PVP components. PVP.Next had to be different, and needed to provide users with a unified experience, regardless of which component or application they were using within the PVP.Next platform.
Enable a multistep, multiyear platform redesign to mitigate impact on partners
The PVP.Next team knew that a complete, single-phase migration from the preexisting PVP to PVP.Next was not possible. Significant downtime was not acceptable, because the PVP user base was too large and the services that it provided were too critical to the Microsoft business model. Therefore, the team planned to implement PVP.Next in a modular fashion, over an extended period, so that it could assess the migration results properly, and ensure that the migration did not compromise the user experience in any way.
Meeting Design Goals by Using Windows Azure
The Windows Azure platform was identified as a suitable candidate to host PVP.Next, based on:
- The design goals for PVP.Next.
- The technology on which the preexisting PVP solution was built.
- How the broad set of Windows Azure capabilities supported the design goals and the technology base.
The Windows Azure platform, with its cloud-based computing architecture, contained many functionality aspects that suited the team's design goals for PVP.Next.
Scalability and Agility
At the core of Windows Azure functionality are roles, which are units of discreet computing resources that IT personnel can allocate to an application or solution. These roles include:
- The Web role, which hosts web-based applications through Internet Information Server (IIS). Its primary design is to host interactive components of an application's interface.
- The Worker role, which typically hosts underlying aspects of application functionality, including background processes, workflow components, or application-related services.
An enterprise can implement these roles in multiple instances and combine them to provide a pooled-resource for the application component that the Web or Worker role is hosting.
A great benefit of instance-based roles is the flexibility they enable for application scalability. Resources can be added or removed from an application simply by adding or removing instances. As a result, Windows Azure-based applications can scale to meet user demand.
In Windows Azure, an enterprise can add role instances to an application very quickly and efficiently. In fact, it often reduces weeks of implementation time to 20 or 30 minutes. In the same manner, instances can be removed from an application in idle or low-traffic periods. Because the Windows Azure customer pays on a per-instance basis, the scalability of instances also contributes to a more cost-effective use of application resources.
Instance scalability also enables Windows Azure to respond better to unplanned or large increases in application demand. When an on-premises application encounters a sudden increase in demand, which exceeds expectations, the on-premises application cannot allocate more resources without a lengthy procurement and implementation process. Often, this process is completed long after the resources are needed.
However, Windows Azure enables IT personnel to assign the extra instance to an application in minutes, rather than weeks, and then remove it when the increased demand subsides.
The Windows Azure design for roles and instances enables individual components within an application to be separated logically. This reduces deployment and maintenance requirements, but still keeps the components integrated fully at the application level.
The PVP.Next team identified that this was a vital capability to allow sharing of different aspects of PVP.Next functionality with multiple applications. By encapsulating a PVP component within a Worker role, the component can integrate easily with other components that the Windows Azure platform is hosting. Additionally, the enterprise could maintain and upgrade components individually, without having to perform maintenance on the rest of the application environment.
Integrated data management
The Windows Azure platform includes SQL Azure™, which provides support for complex data structures and data management within the cloud-based architecture. Because most of the preexisting PVP data storage was in on-premises SQL Server databases, the PVP.Next team knew they could reuse many data-storage components.
Once the team determined that Windows Azure was the target platform, the team was ready to begin migrating PVP to the Windows Azure platform.
The first and most obvious architecture decision was that the migration would not be an all-or-nothing implementation. One of the goals the PVP.Next team wanted to achieve was to provide the same or better set of service-level agreements (SLAs) to their business partners, while simultaneously adding new capabilities at a lower cost. This proved to be a challenge. The team had to balance the integration of existing services with newly created Windows Azure Platform services that ensured a seamless integration between cloud and on-premises resources.
The team recognized that both PVP and PVP.Next would have to coexist during the multiyear migration process. Therefore, the team outlined components to be migrated, established milestones, and then began the process of implementing PVP.Next as a hybrid solution.
Implementing a Hybrid Solution
Hybrid solutions are common in many applications based on Windows Azure. In a hybrid solution, Windows Azure hosts and manages portions of the application, while other components remain on-premises in traditional data centers. In the case of PVP, the solution's scale did not allow for an immediate and complete migration. As a result, the hybrid architecture was implemented to allow for a smooth transition between on-premises and Windows Azure, until the migration is complete. In other cases, on-premises components may remain as part of the application architecture because of business requirements, security limitations, or to support other on-premises applications.
Migrating Data Structures
The implementation of PVP.Next began with its most critical component, the PVP databases. The PVP databases held the core of PVP information and business logic, and were critical to the functionality of PVP applications. However, the actual design of the databases did not support a logical workflow and data integration between applications. The PVP databases were hampered by a number of design flaws, which were the result of many years of application integration and functionality growth, coupled with limited planning. Therefore, the team began a redesign of the database structure.
By starting with the databases, the PVP.Next team had adopted an inside-out strategy for the migration to Azure. They started with the core of PVP--the databases--and then worked outward from there. This outward migration included the integration of critical solution-wide processes and high-impact components, and continued to include individual applications and noncritical functionality.
The three core components of the database--Assets, Requirements and Enrollments--were assessed individually based on their structure and suitability for SQL Azure or other Windows Azure-based data storage.
Leveraging Windows Azure Table Storage
The Requirements and Enrollments database suited a SQL Azure-based deployment, but the team realized that several aspects of the Assets database made it a better candidate for utilizing data-storage and management tool: Windows Azure Table Storage.
Windows Azure Table Storage provides a collection of row-like entities that can contain up to 255 properties, much like a SQL Azure database. However, the structure of Windows Azure Table Storage is not constrained by a schema that defines the values that each field can hold, nor does it maintain relationships that interconnect data. Additionally, cost per megabyte (MB) for Windows Azure Table Storage is significantly lower than the cost of storing data within a SQL Azure database.
Because of the considerable size of the Assets database, and the lack of requirement for relational data within it, the PVP.Next team migrated the Assets database to Windows Azure Table Storage.
Managing Data Between On-Premises and Windows Azure Components
Once the data-design decisions were made, the team needed to find a way to enable coexistence between on-premises databases and Windows Azure-based databases. The databases needed to maintain the existing data so that their corresponding applications could continue to access it, but also needed to ensure that the data being modified within the databases was synchronized between the two infrastructures.
The PVP.Next team implemented several components to enable data synchronization and ensure consistency between on-premises and Windows Azure-based databases.
First, the on-premises databases were maintained in order to support any aspects of PVP.Next that remained on-premises. As components were migrated to Windows Azure, the corresponding data components were moved to Windows Azure Table Storage (Assets database) or SQL Azure (Enrollments and Requirements databases).
An on-premises service (Asset Delta Service) was created to assess changes made in the on-premises Assets database, and then communicate those changes to its partner service in Windows Azure, the Asset Odata Web Service. The latter would ensure that the changes were reflected in the Assets data structure in the Windows Azure Table Storage.
Synchronization of the data between the Enrollment and Requirements databases was performed by using Windows Azure Worker roles in combination with on-premises SQL Server Integration Services (SSIS) packages. This ensured the proper preparation of the data prior to synchronization, and eliminated the possibility of conflicting data, or data that did not synchronize correctly, in the on-premises and Windows Azure-based data components.
Monitoring a Hybrid Solution
The PVP.Next team understood that monitoring the new application would be a challenge. They worked closely with the operations team from the beginning of the implementation to ensure that deployment, monitoring, and maintenance occurred in an effective, efficient manner. The team leaned heavily on Microsoft System Center Operations Manager to enable monitoring across the entire PVP.Next platform. This provided visibility into both the on-premises and Windows Azure environments, through the Windows Azure Management Pack and additional custom-designed monitoring structures.
Current Solution Architecture
The following figure represents the current state of PVP.Next architecture that has migrated to the Windows Azure platform. As noted earlier, subsequent phases of the PVP.Next implementation will migrate the rest of the PVP infrastructure to Windows Azure-hosted cloud-computing components.
Figure 1. Current PVP.Next Architecture Hosted on Windows Azure
While the implementation of PVP.Next is an ongoing project for Microsoft IT, they have already realized several benefits from the migration to Windows Azure:
Performance and Scalability
The solution delivers virtually unlimited scalability, and it increased availability significantly, at a lower cost when compared to the on-premises solution. Even when the service experiences larger-than-normal traffic load, the average response time is well within the team’s SLAs. Furthermore, the team has not experienced any uncompleted requests.
One of the most important goals of the PVP.Next design was to enable cost savings and increase business value. The migration to PVP.Next on Windows Azure has resulted in the following cost savings:
- A greater share of IT investments are being directed from capital expenditures and maintenance to operations and innovation initiatives.
- Development of new programs has been reduced from hundreds of thousands of dollars to between $10,000 and $25,000 U.S. dollars.
- While maintaining a hybrid solution represents both on-premises and Windows Azure costs, the PVP.Next team still anticipates a 72 percent increase in business enablement and innovation investment, and a 21 percent reduction in annual maintenance costs. This could result in a net savings of approximately $4.3 million U.S. dollars during the next three years.
The development environment for the Windows Azure platform, as well as the instance and role-based infrastructure that Windows Azure provides gave PVP.Next agility that could not have been experienced with the previous solution. Upgrades and environment changes occur in a matter of minutes, rather than weeks. A team now can move test environments in which it is working on new releases into production with a just few clicks in the development interface. The previous solution required a laborious manual cutover.
Robust and Familiar Development Tools
Windows Azure supports open standards, and it utilizes familiar tools, such as the Microsoft Visual Studio® development system and the System Center products. This enabled the PVP.Next team to quickly design, develop, and deploy a highly interoperable REST service that has the performance and on-demand scale that MPN requires. Because Windows Azure is so prescriptive, many of the architectural decisions already have occurred. This may not be appropriate for every software solution, but it worked very well for the PVP.Next team.
Because Windows Azure packages and deploys the entire solution in a consistent environment, it prevents the time-consuming bugs that configuration errors and differences introduce. The team did not experience many of the typical configuration issues that occur during traditional development, even as it added new personnel, and as it moved from development to testing, and then to preproduction and production.
Creation of Unique Environments
The PVP.Next team found that it is very fast, easy, and inexpensive to create unique environments (development, test, integration, and preproduction) by using Windows Azure. When the team is finished with an environment, it can let it go. When the team wants the environment back, it can recreate it very quickly. This reduces the cycle time for ramping up and down.
The PVP.Next team also found that Windows Azure makes it easy to create multiple proof-of-concept environments that greatly improve user feedback. The team created several demonstrations for business partners, and received much better feedback from those demonstrations than it had from the functional specifications.
Deployment Automation and Testing
Windows Azure provides an application programming interface (API) and scripts that fully automate deployment, which reduces human error during the deployment process. Due to the standardization on Windows Azure, the PVP.Next team was able to test its production-deployment processes and scripts throughout the software-development life cycle. This greatly reduced the last-minute issues that an enterprise often encounters during production deployment.
Writing code for Windows Azure is very familiar for developers who are accustomed to working with Visual Studio and Microsoft .NET. However, there are some important differences. Namely, the PVP.Next team learned that it needed to:
- Pick the right application. The primary key to success in developing for Windows Azure is to pick the right application. Windows Azure is great for building a web application or a compute-intensive application, but it is not yet a development platform for general-purpose applications. The PVP.Next team started with a lower-risk, customer-facing project to validate that everything worked as planned.
- Prepare for the impact on operations. During development for Windows Azure, operations undergoes the biggest change. It is critical to understand the impact to operations, get the operations team on board early, and design for operations. Because the PVP.Next team knew that it faced an unknown operations environment, the team instrumented its code heavily so that it would know exactly how many transactions succeeded or failed. The team took advantage of the out-of-the-box Windows Azure diagnostics to do the bulk of the work, but also used third-party tools.
- Prepare early for security and integration. Security and integration are very important considerations. Active Directory® Federation Services (AD FS) is a great security solution for authentication. Integration is challenging if there is data on both sides of the firewall. Therefore, for projects that require integration, it is important to make sure that an enterprise solves security and integration issues before starting development.
- Build in SQL Azure retries. SQL Azure moves databases to balance load. When a database is moved, the connection pool becomes invalid. If the service does not retry the SQL connection, the connect request fails, which causes errors. Therefore, it is critical to build in SQL Azure retries.
- Conduct performance testing. Running performance and stress tests with Visual Studio and Windows Azure can be challenging. The problem is the wide area network (WAN) link between the client and the server. Results may be skewed, and the network can become a bottleneck quickly. The potential to accumulate bandwidth costs also exists. It is wise to create a simple application that it deploys in Windows Azure, in the same data center, so that heavy testing on the same network can occur. This type of testing provides feedback on real performance data.
The PVP.Next team at Microsoft IT created a multitenant, hybrid application based on Windows Azure, which enabled Microsoft partners to learn about opportunities to strengthen their technical capabilities, gain expertise to help serve customers better, and be part of a community that sparks innovation and connection. The PVP.Next team used familiar development tools to quickly create a Windows Azure-based solution that has extremely high availability, scalability, and performance. The new solution also provides significant cost savings for MSIT and the enterprise, and it will continue to grow as the migration to Windows Azure continues.
For More Information
For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Order Centre at (800) 933-4750. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information via the World Wide Web, go to:
© 2012 Microsoft Corporation. All rights reserved.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Active Directory Federation Services, Microsoft SQL Server, Microsoft Visual Studio 2010, SQL Azure, Windows, Windows Azure, Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.