Microsoft Individual & Organization: Master Data Management at Microsoft

Technical Case Study

Published: August 2008

In a large organization, different divisions and business areas may maintain their own versions of customer data. The data is frequently business critical, such as proprietary customer lists, prospective customer lists, and product information. For example, one division might track customer orders and therefore have shipping and billing information for the customer, while another division has records pertaining to the organizational structure and hierarchy for that same customer. The data itself is not synchronized between systems. Over time, these systems continue to gather customer information (current and historical) that provides only a slice of the whole customer story.

Master Data Management enables companies to integrate and synchronize data from multiple sources into one authoritative source to which applications can refer. Microsoft Information Technology (Microsoft IT) used MDM concepts to implement the Microsoft Individual & Organization initiative, which helps solve the problems of multiple data sources and inconsistent data. MIO is the single source of core customer data for Microsoft.

This paper examines the architecture of MIO, how MIO is integrated with other applications, and how Microsoft uses MIO. This paper is intended for the chief information officers, information architects, and business decision makers.

Situation

Organizations find that maintaining consistency in customer information and contact points is difficult, especially in today's IT environment. In many cases, the various touch points that a company like Microsoft has with a customer are gathered and supported by their own IT systems, and these systems are not build to share data. All this leads to data inconsistency and ultimately to an inconsistent experience for the customer, or even for internal users who try to contact and work with that customer.

The business impacts of inconsistent data can be significant. Frustrated customers can lead to lost sales opportunities and lost revenue.

Solution

MIO gathers data from multiple sources and creates a single data hub that consuming applications then use. It creates an environment in which customers and partners are identified correctly and consistently across the entire Microsoft organization. It defines and maintains customers' contact preferences (for example, privacy preferences), and it records their transactions. MIO integrates the user experience regardless of the touch point in which the customer or partner contacts Microsoft.

The solution has improved:

• Customer satisfaction. Accurate contact preferences reduce the potential for inconveniencing customers. Also, a consistent set of data helps enable consistent customer experiences.
• Productivity for sales, support, and operations. Staff have confidence that the data is accurate, so they can focus on managing customer relationships.
• Reporting. MIO improves the accuracy of reports that use customer data. For example, the authoritative data in MIO helps improve the Customer License Position (CLP) and Territory Management System (TMS) reports.

MIO Database

Building the MIO database involves creation of a data model, or a view of the data as it is structured, integration of data from multiple sources, and sorting through that data to match similar information. At times, data sources conflict, which creates a challenge in deciding which source to trust.

Data sources include customer-management applications at Microsoft matched against data from third-party sources such as Dun & Bradstreet (D&B). MIO contains two basic types of customer data: organization and individual. MIO uses D&B as its primary organization source, whereas lists of contacts for people, including those from third parties, are used as individual sources.

Part of the challenge of building a centralized data hub involves reconciling different data models as implemented in diverse applications.

Data Model

Early on in the MIO initiative, Microsoft IT identified the need for the various IT systems to agree on a common data model. The data model for MIO is divided into five subject areas:

• External/address. Entities related to the purchase of non-Microsoft �firmagraphic�/demographic data.
• Organization. Entities related to organizations, along with any attributes that Microsoft assigns for those organizations (their industry category and market segment).
• Contact preference. Methods used to contact individuals and organizations, such as postal addresses, e-mail addresses, and telephone numbers.
• Persona. The name and primary identifying information that an individual provides for conducting business with Microsoft.
• Individual. Real-world individuals and information about their relationships with organizations.

These five subject areas relate to other functional areas within Microsoft (sales, marketing, product management, agreement management, and incident management), as shown in Figure 1.

Figure 1. MIO subject areas as related to functional areas

It is important to note that MIO centralizes only data that is shared across multiple applications. Mission-specific data�for example, partner benefits�remains in the relevant application (in this case, the partner management application). However, when referring to a particular partner's address, the application will use data centralized in MIO.

Integration of Data

As data arrives from sources such as D&B and older applications, it must be analyzed and coordinated so that the entity is represented only once and correctly. Some issues faced in this area include determination of the freshness of the source data, the differing ways in which an entity can be represented, and the level to which a given source can be trusted.

Matching

Customer data changes frequently. Contact people at organizations move to different roles or different organizations; telephone numbers and e-mail addresses also change over time. The data source must be updated to match the new information about the entity. When an organization uses multiple repositories to store customer information, the data falls out of synchronization quite easily. An MDM system such as MIO solves this problem. Providing a single source of data means that customer information is updated once in a single repository.

Additionally, each source of data has its own representation of a given entity. Although sometimes the data is categorized based on a common key, such as the D&B number, it is more frequent to find custom keys used to represent the same customer in different systems. Even when a common key is used between two data sources, the information contained in the record may differ slightly. For example, an organization may be known as Northwind Traders in one database and simply as NWT in another. Matching these records is a challenge.

Compositing

When multiple sources exist for the same record and those sources conflict, the organization must choose one source. Determining which source to trust is another challenge in building an MDM application such as MIO. Sources such as D&B would be trusted more than internal, older customer-management applications that have outdated and unmaintained data.

The product of overcoming matching and data-trust challenges is a composite matching result�the final set of data placed in MIO for a particular entity. It represents the best-effort match, factoring in issues such as the source of the data and its age. Where MIO cannot find a match, it requires human intervention to resolve the conflicts.

Figure 2 provides an overall view of data entering the composite matching result (called the master data composite in the figure). In this figure, each data source has information about a particular organization and therefore has candidate information about that entity. Candidate �D&B� (Dun & Bradstreet) and Candidate �+1� (which represents a third-party data source used with D&B) are both considered third-party data sources. Candidate �MSO,� or older sales data, is considered consolidated legacy data. Candidate �PMC� represents line-of-business (LOB) applications, which may also be sources of data for MIO.

Figure 2. Data from multiple sources is integrated into one master data composite

While data is being placed into MIO, its quality is improved through further validity checks�for example, address standardization. These data-quality enhancements are applied through Web services and other integration interfaces.

Additionally, one of the most important aspects of any MDM application is its ability to retain the history of the source data so that the data can be pulled back to its original component s. MIO merges records logically to create a physically merged record while maintaining the original source records.

System Architecture

Figure 3 illustrates the system architecture, including the flow of data from its source through MIO matching services and finally into MIO. Click Figure 3 to view larger version.

Figure 3. The MIO architecture as it relates to data synchronization

MIO Application Models

Applications within Microsoft interact with MIO to obtain authoritative customer data. MIO offers two primary methods for consuming the data: Web services and batch transfer.

Integration Through Web Services and BizTalk Server

For high-performance transactions, MIO offers a set of Web services that enable search, lookup, creation, update, and deletion of customer records contained in MIO. These two-way Web services are exposed through the Windows® Communication Framework:

• Inbound Web services, used to search and update MIO:
  • Synchronous Hypertext Transfer Protocol over Secure Sockets Layer (HTTPS) for searching and lookups
  • Asynchronous Message Queuing (also known as MSMQ) service for creation, deletion, and update
  • Asynchronous Message Queuing service for event acknowledgments
• Outbound Message Queuing service for events:
  • Real-time events for creation
  • Periodic events for synchronization

More complex integration with MIO is achieved through the Microsoft BizTalk® Server platform. For example, the BizTalk Siebel Adapter enables integration into MIO through BizTalk.

Batch Data Transfer

Batch or bulk transfers of data are available as part of MIO. These are used for data warehouses, data marts, and other operations such as delta file processing and MIO data services. Delta file processing refers to the periodic bulk loading of third-party data; MIO data services refers to bulk matching and bulk updating.

Integration of New Applications and Their Data

Microsoft benefits greatly from having MIO manage its data. As application data sources are identified as possible candidates to have their data managed by MIO, they are integrated. The integration, or on-boarding, of new application data into MIO is where application models come into play. These applications are known as On-boarding Applications (OBAs). There are three primary steps, each with several smaller steps involved in the on-boarding process. These three steps are sometimes preceded by a proof of concept that outlines the more difficult scenarios involved in integration of that application. The three steps are:

1. Preparing to onboard to MIO. This step determines:
   • What data from the source application MIO will master.
   • The level of data alignment appropriate for the application.
   • Which MIO user scenarios will work best with the application.
   • The integration user experience for the application.
   • Which business processes will change when integration into MIO is complete.
   • What changes will be necessary for the application to work with MIO.
2. Establishing the initial synchronization with MIO. This step includes:
   1. Identify records of interest.
   2. Detect duplicate records by using MIO data services.
   3. Resolve ambiguities between the application and MIO.
   4. Assign �name-only� matches to their appropriate location within the data hierarchy.
   5. Assign the application's parent matches to the appropriate location within the data hierarchy.
   6. Verify stand-alone records.
   7. Execute the initial data load.
3. Establishing the ongoing synchronization with MIO. This step includes:
   1. Determine the appropriate method for maintaining data synchronization: general framework methods, MIO clients and proxies, eBiz-hosted BizTalk Server environment, or eBiz Firewall Broker and BizTalk Server.
   2. Determine which, if any, mandatory items to store locally.
   3. Create an application-initiated creation process.
   4. Create an application-initiated update and delete process.
   5. Create an MIO-initiated update and delete process.
   6. Make MIO updates visible to users.

Microsoft IT identified the following criteria that an application must meet to be fully on-boarded with MIO:

• All OBA records of interest must have identical content to that of MIO.
• All OBA records of interest can be consistently exchanged between MIO and the OBA, both ways, without loss of data.
• All OBA records of interest are correlated with only one MIO identification number (ID).
• All OBA records of interest have the corresponding MIO ID and MIO version ID stored in the OBA application or can derive the MIO ID from the record.
• All relevant OBA processes search on MIO prior to creation of the local system account.
• All OBA records of interest in synchronization with MIO are aligned with MIO business processes to remain consistent.
• MIO receives all OBA account creations and updates within five minutes.
• There is a measurable improvement in an OBA difficulty, as determined by data quality diagnostics before on-boarding.

Current Position and Future Benefits

The goal of any MDM platform is to provide the single source of authoritative data for the data that it manages. MIO does exactly that but also provides much more through its integration points and ability to quickly on-board other applications.

MIO is an active and dynamic application with momentum and support to achieve its goals. The program is in its third year and continues to improve. Previous achievements include the management of more than 80,000,000 records, including their legal hierarchies, bulk matching and importing, Web service integration, proactive data quality, and a full software development kit (SDK) for on-boarding.

Recent achievements and events related to MIO include:

• Integration with Agent Desktop. With the connection of Agent Desktop, MIO has now connected service request and call-center data about interactions with Microsoft customers. Integration between this application and a sales application (through MIO) enables a call-center agent to enter a lead in the Microsoft customer relationship management (CRM) system without leaving his or her call-center application.
• The on-boarding of Siebel. This additional source of data furthers the value of MIO at Microsoft.

The fiscal year 2009 (FY09) release of MIO will focus on performance and scale. MIO will seek to connect systems and optimize the platform to become the kernel of the customer ecosystem. MIO provides a meta-directory of customer information, helping to form the hub of a customer relationship platform that will unite access to customer data.

MIO is also a pillar in the privacy strategy at Microsoft, because MIO can provide a single, authoritative master for individual contact preferences such as Do Not Call records. All secondary-use systems, such as campaign management, will use MIO.

Within these goals are a partial 360-degree view through the MIO-connected applications. Microsoft IT is implementing country-specific match tuning within the services area. Microsoft IT is improving the foundation of MIO with the use of a highly scalable infrastructure.

In FY09, MIO will also offer enterprise data warehousing (EDW) integration and improved partner data quality.

Conclusion

MIO represents a singular source of enterprise data related to Microsoft customers and partners. It is an implementation of MDM at Microsoft and is used to solve the problems related to multiple sources of data. MIO is already providing great value for Microsoft by reducing difficulties in the management of multiple sources of data. Not only do these improvements lead to savings in terms of time spent managing customer records, but MIO also enables more efficient sales and marketing efforts by organizing customer preferences and contact information.

For More Information

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information via the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase

 

© 2008 Microsoft Corporation. All rights reserved.

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, BizTalk, SharePoint, SQL Server, Visual Studio, and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.