How Microsoft Information Technology Used Microsoft Forefront Identity Manager 2010 R2 to Scale Group Management
Technical Case Study
Published: July 2012
The following content may no longer reflect Microsoft’s current position or infrastructure. This content should be viewed as reference documentation only, to inform IT business decisions within your own company or organization.
Microsoft Information Technology (MSIT) used Microsoft Forefront Identity Manager 2010 R2 to deliver the scale and performance required to improve group management. MSIT used Forefront Identity Manager R2 to provide improved synchronization performance, better object management, and improved data filtering. Additionally, MSIT automated configuration to streamline deployment significantly, and incorporated time-saving custom features.
Technical Case Study, 408 KB, Microsoft Word file
Products & Technologies
The initial release of Microsoft Forefront Identity Manager 2010 enabled Microsoft IT to begin eliminating its custom group-management application. However, the projected growth in adding computer objects created a need to scale the solution to new levels while continuing to meet peak demand performance targets.
Microsoft IT also needed to improve the agility of deploying changes to Microsoft's evolving identity management environment.
Microsoft IT implemented Microsoft Forefront Identity Manager 2010 R2 to utilize product improvements in performance, scale, and analysis.
Additionally, by adopting use of the Forefront Identity Manager Windows PowerShell cmdlets, Microsoft IT was able to streamline deployment of Forefront Identity Manager 2010 R2 and help to ensure minimal effect on enterprise-wide operations..
Forefront Identity Manager 2010 R2 enabled Microsoft identity-management professionals to easily manage identity data that was synchronized across different resources, such as applications, databases, and servers. The product also introduced an array of time-saving and cost-saving user features, such as self-service password resets, which greatly reduced reliance on Help Desk personnel.
However, like many enterprise organizations, Microsoft faced challenges in the ever-changing identity management landscape. As MSIT progressed with enterprise-wide migration to Forefront Identity Manager R2 (FIM), the team recognized that adding hundreds of thousands of computer objects, as members of security groups, to the system would significantly increase the scale of their implementation and stretch synchronization time beyond acceptable levels. Therefore, changes to both FIM and MSIT's implementation of the product were required to meet scale and performance targets.
Evaluating business and operational needs
To identify needed improvements, MSIT worked closely with the FIM Product Group to evaluate projected growth and existing performance to determine where changes would be necessary in the product, its configuration, or both. The results of this evaluation made it clear that the overhead associated with synchronizing data had to be reduced to ensure that performance and scale needs could be met.
To achieve this objective, the teams recognized that MSIT would have to change the way that synchronization policy works, and change how any future revised policy was implemented.
On further evaluation, MSIT identified other desired areas of improvement, including:
- Changing how search scopes work. The solution needed to accelerate queries performed by users of the Forefront Identity Manager portal, and incorporate more search flexibility overall.
- Improving self-management features. The solution needed to reduce the costs associated with managing email and distribution groups.
- Incorporating the ability to implement and deploy solutions in a highly flexible manner. The solution needed to avoid major business disruption.
To address scale and performance requirements, MSIT began developing its solution by first conducting a thorough review of the existing configuration. This review found that a significant portion of the objects in MSIT's FIM implementation were used for synchronization, which effectively reduced MSIT's ability to scale up. Therefore, to resolve the scale and performance issues, MSIT had to first evaluate the role that these synchronization objects performed within the system.
FIM utilizes what is known as an outbound synchronization policy. That policy relies on sets, management policies, and workflows to apply an outbound synchronization rule. When a rule that is based on outbound synchronization policy is applied to a group, FIM also creates another object called an expected rule entry (ERE), and associates it by reference with the group object.
When an import/synchronization cycle between the FIM Service and the FIM Synchronization service is run, this one-to-one relationship of a group and its ERE object flows from the FIM Service into the FIM Synchronization service. This resulting process creates substantial overhead in the system for an enterprise the size of Microsoft that comprises more than one million objects. Synchronization, as previously structured, made it extremely difficult to manage computer objects on an as-needed basis.
Recognizing the overhead that outbound synchronization policy would place on the system once computer objects were introduced, MSIT decided to replace that configuration with a new FIM 2010 R2 feature, Filter-Based Outbound Synchronization Rules. This filtering feature provides substantial scale and performance benefits by using a one-to-many model, instead of the one-to-one model that outbound synchronization policy uses.
MSIT identified many additional issues that it sought to address with the upgrade; however, the team chose to first focus on the performance and scale issues that would arise with the migration of computer objects into Forefront Identity Manager.
To accomplish this objective and streamline the addition of computer objects, MSIT devised a plan that included the following steps:
- Create any necessary attributes to enable the FIM Synchronization service to determine the appropriate means of provisioning--creating, modifying, and retiring--identities.
- Change existing workflows to populate helper attributes.
- Delete EREs.
- Replace existing outbound synchronization policy rules with filter-based outbound synchronization rules.
- Remove unnecessary workflows.
MSIT recognized that deploying numerous manual configuration changes would require an evolution in the process for engineering, testing, and deploying Forefront Identity Manager. The objective was to reduce reliance on the largely manual and historically tedious process that was used to apply documented configuration in previous deployments. To achieve this objective, the team decided to use FIM cmdlets through the Windows PowerShell® command-line interface as a foundation for automation.
Rethinking and Revising FIM Configuration
Other configuration modifications were required to support the switch from outbound synchronization policy to filter-based outbound synchronization rules. When MSIT used outbound synchronization policy, rules were applied based on a group's membership in a set that corresponded to the group's respective Active Directory® Domain Services (AD DS) forest.
As Microsoft has multiple AD DS forests, some of which have multiple domains, membership in the correct set had to be calculated using criteria that examined a group's domain attribute, and then performed an "OR" operation among many domains to calculate the forest. However, filter-based synchronization rules do not rely on sets, so there is no equivalent OR operator for scoping filters. As such, MSIT had to devise a new way to present this value to the FIM Synchronization service so that it could determine which synchronization policy to apply.
To resolve this, a new forest attribute was created in both the service and sync schemas so that each group's forest name could be stored. A transition-in management policy rule was created and associated with the set. This policy rule triggered execution of a workflow activity whenever a new group joins the set. When executed, the workflow function activity now adds a value to the forest attribute on the group.
To apply this policy retroactively to existing groups in Forefront Identity Manager, MSIT used the Run On Policy Update (ROPU) option. This option was improved in Forefront Identity Manager R2 by enabling administrators to adjust the configuration to meet specific environment processing conditions.
When making the switch to filter-based outbound synchronization rules, a temporary service outage for users was required. The outage was performed to ensure that no new, unapplied EREs remained in the system prior to deletion. Once interactive and programmatic access had been disabled, MSIT performed data synchronization between Forefront Identity Manager and AD DS to complete any unapplied EREs in the system.
MSIT then ran a script to remove almost two million EREs from the FIM Service database. On completion, a Windows PowerShell script was run to add the new filter-based outbound synchronization rules, update existing inbound sync rules, and then remove unnecessary outbound synchronization policy rules, management policy rules, and workflows.
After completing the configuration changes, MSIT ran a required full import and synchronization. After completion of these tasks, both the total object count and the time required to perform a full synchronization had been reduced by more than half.
Although this modification reduced object counts and gave MSIT the capacity to bring in computer objects at scale, additional work was required to help improve performance. MSIT targeted two particular areas:
- Export to Forefront Identity Manager
- Import and synchronization from AD DS
To improve export performance, MSIT and the product group used the newly developed FIM 2010 R2 management agent asynchronous export modes. Before these modes were implemented, the benchmarked maximum AD DS–to–FIM export rate for MSIT was approximately 4,000 objects per hour.
MSIT then tested export to FIM using MA asynchronous single-object mode. This mode, in conjunction with a newly created Forefront Identity Manager service-configuration parameter for the maximum number of concurrent synchronization export requests, increased the export rate to 8,000 objects per hour. Following the single-mode test, MSIT tested export using a batch mode. The result of this test was a dramatic improvement: 53,000 objects per hour.
Addressing Active Directory Synchronization Performance
MSIT also sought to reduce the synchronization time that is required after importing data from AD DS. To accomplish this, MSIT used a new feature, the Active Directory connector import filter. This filter enables administrators to configure the Active Directory management agent to identify and exclude specified objects from the connected space during import.
Prior to the FIM 2010 R2 update, all Active Directory objects were brought into the connector space during an import, and therefore had to be evaluated during synchronization. This caused the FIM Synchronization service to waste valuable time evaluating objects that would never be joined to the metaverse data layer. It also meant that objects that were otherwise unused were taking up room in the connected space.
The relative value of a connector import filter will depend on organizations' individual needs, as there is a trade-off between increasing import times to enable the additional object evaluation, and reducing synchronization times. For MSIT, the slightly longer import times were offset by the overall improved performance.
Improving Search Scope and Functionality
The Forefront Identity Manager product group helped MSIT implement an important and useful change in the how FIM's default search scopes operate. Before MSIT implemented computer objects, queries that were initiated at the portal were returned in a timely manner. As the Identity Manager service database grew, however, query times increased. This occurred because of the Identity Manager default search scope behavior, which uses the XPath Contains() function to look for a DisplayName lookups.
In FIM 2010 R2, the Starts-With() function replaces the Contains() function in DisplayName lookups, which significantly improves lookup performance. This is because the Starts-With() function can optimize the full-text data indexing on DisplayName lookups. In addition, with searches using a single prefix term, any word starting with the specified term is included in the result set. For example, the single prefix term bene matches benefit and beneficial.
Two new attributes were also added in FIM 2010 R2:
- An AdvancedFilter attribute lets users specify the desired filter query.
- A DefaultSearchScopeName attribute allows users to specify the desired search scope for a UocIdentityPicker.
Automating FIM 2010 R2 Configuration
MSIT also decided to automate the FIM 2010 R2 deployment. In prior deployments, the team had relied largely on documented configuration being applied manually. This process was tedious, lacked reliability, and hindered agility.
To reduce these challenges, the MSIT team decided to utilize the Windows PowerShell cmdlets that are part of FIM configuration-migration toolkit installed with the FIM service. Although the cmdlets were originally intended for migrating an existing FIM service configuration from one system to another, MSIT discovered that they could also use the cmdlets to apply customizations after installation.
Using the Export-FimConfig and Import-FimConfig cmdlets, an administrator could configure most get-configuration and set-configuration changes required for FIM deployment or maintenance. For example, when MSIT replaced its existing outbound synchronization rules with the FIM R2 rules based on filters, the entire operation was created, tested, and implemented using Identity Manager's automation scripts.
Although MSIT incurred additional costs in scripting these solutions, those costs were offset ultimately by the benefit that automation provided through repeatability, reliability, and agility. To further control costs and simplify implementations, MSIT implemented its scripts using Windows PowerShell 2.0 modules. Windows PowerShell 2.0 provided the ability to group and save functions into a single file called a module. By utilizing modules, common Forefront Identity Manager configuration actions were abstracted as functions, thereby enabling engineers to share and reuse code when creating their specific configurations.
Group life-cycle management (GLCM), noted as GLCM for the purposes of this document, is a new feature that MSIT created. Essentially, MSIT created GLCM to help maintain the health of the Forefront Identity Manager environment by eliminating groups that are no longer in use. GLCM accomplishes this task by managing each group object's expiration date in FIM.
A group's ExpirationTime can be updated either explicitly or implicitly by the group's owner. Explicit extension of a group's ExpirationTime occurs when an owner selects a specific user interface option to extend the group's lifetime. Implicit extension of a group's ExpirationTime can occur any time an owner extends the group as the result of normal maintenance activities.
To set and track a group's object lifetime (ExpirationTime), MSIT created an executable that works with Identity Manager. This executable runs as a scheduled task on a recurring basis to set a group's initial ExpirationTime value. In addition, this executable also updates the value if a group owner has implicitly or explicitly asked for an ExpirationTime extension, and deletes groups that have reached the end of their lifetime.
MSIT implemented the implicit trigger, to notify group owners of impending group deletion, in addition to the ability to explicitly extend a group's lifetime, to minimize the possibility that an active group might expire or ultimately be deleted. However, neither tactic entirely eliminates the risk that some groups might still unintentionally (from the owner's standpoint) expire or be deleted.
For example, consider the scenario in which groups are created with join restrictions of either Owner Notify or None. In either of these configurations, no owner interaction is required for users to join or leave the group. As such, these types of groups may go dormant by an owner for extended periods, so the implicit extension of their ExpirationTime would not be triggered. If the pending expiration was also overlooked, a valid group might be deleted.
To address this potential problem and reduce requests to restore expired groups, MSIT incorporated the new FIM 2010 R2 blind carbon copy (BCC) email notification feature as part of its strategy for informing multiple group owners when their group's expiration time is nearing. As groups reach milestone dates prior to expiration, they become members of sets that have policy applied to trigger workflow notification activities. For the earliest of these notifications, MSIT wanted to limit the number of individual notifications, to avoid affecting system performance.
By using the new BCC capability, MSIT is able to create and send a single generic notification to all qualifying group owners. When a group owner receives this notification, the group owner can click on a link in the email that launches a webpage browser displaying their owned groups. From this page, group owners can choose to extend the expiration time of their group(s), as show in Figure 1.
Figure 1. Expiration time extension webpage
In addition to keeping the overall number of objects in Forefront Identity Manager to a minimum, GLCM also provides the means to restore objects that have been removed. For MSIT, the additional benefit of GLCM is that it also reduces synchronization times and security risk.
MSIT also created a way to restore groups that were deleted because their expiration time had not been extended. This solution utilizes the Active Directory Recycle Bin ability to retain deleted objects for 180 days after those objects have been deleted in FIM.
The challenge of returning a group to its prior state in Identity Manager from the object in AD DS is complicated, as many attributes of a group are not typically flowed to the corresponding object in AD DS. To resolve this problem, MSIT created a management policy rule for delete requests that applies an Authorization (AuthZ) custom workflow activity. When the Delete Group AuthZ workflow activity runs, it performs a lookup of the delete request target group. The activity reads the attributes of the group in FIM that are not part of the Active Directory object, aggregates this data, and then writes it directly to a previously unused extensionAttribute of the object in AD DS.
Upon successfully performing this work, Identity Manager is permitted to delete the group; following the next synchronization cycle, AD DS moves the group object to the Recycle Bin. At the end of the retention period, if an owner has not requested restoration of the group, AD DS scavenges it. However, if an owner makes a request to have a group restored before AD DS scavenges it, all required attributes for the group can be restored from the Active Directory group object.
Using a tool written by MSIT, support personnel can initiate a targeted group's restoration at an owner's request. The utility simultaneously modifies an extensionAttribute of the restored group object in AD DS that signals Identity Manager that the group needs to be restored as part of the synchronization cycle.
When the group is exported back to Identity Manager, it becomes a member of a special set according to criteria MSIT created in Identity Manager. Membership in this set triggers a transition-in restore group management-policy rule that applies a workflow activity. This workflow activity reads the object's extensionAttribute in AD DS, which contains the unique Identity Manager data that was stored by the prior deletion activity. Once the restore workflow activity applies the data back to the respective attributes of the group in Identity Manager, it clears the value of the extensionAttribute on the object in AD DS.
- Identify business rules and requirements. IT teams should clearly define and document the project's overall scope, and the desired features, improvements, and functionality.
- Determine optimal deployment strategy. To plan for minimal possible disruption, organizations should evaluate the relative risks and benefits of a one-time versus an incremental upgrade to FIM 2010 R2. If a phased deployment is chosen, using virtual machines for loading and test staging environments may help minimize hardware costs, and ease setup and restoration of environments as resource requirements fluctuate.
- Plan deployment well ahead to avoid disruptions. This may require configuring rule changes and identifying potential effects on critical business units to accommodate short-term performance slowdowns and minimize unnecessary re-synchronization. Creating a test plan will help verify that business requirements and scenarios work as intended.
- Use a source-control system. This approach helps accommodate frequently changing scripts and unanticipated scope-modification needs after the deployment is underway.
- Take advantage of Microsoft Visual Studio® Team Foundation Server 2010. The tools in Visual Studio Team Foundation Server 2010, or other collaboration and development tools, enable engineering teams to work in a logical, cohesive manner while defining project scope and orchestrating implementation and deployment.
- Plan for proactive ongoing management post-implementation. To maintain upgrade benefits and improved performance, ensure ongoing active management of FIM 2010 R2 in the enterprise, especially in the areas of group life-cycle management.
- Maximize the benefits of the upgrade effort. Identify business process improvements, customized functionality, and emerging organizational needs that might be addressed in tandem with an upgrade to Forefront Identity Manager R2.
- Improved performance, faster synchronization, and reduced server usage. MSIT achieved a six-fold increase in the object-export rate during synchronization, growing from 10,000 to approximately 60,000 objects exported per hour. Database object-capacity increased by 100 percent as well, to more than two million objects stored, including computer objects.
- Increased configuration flexibility to meet the needs of individual business units or departments, and the ability to set custom UI parameters.
- Reduced reliance on administrators and Help Desk through enhanced self-management tools. Some group parameters and associated tasks previously requiring administrator activity are now performed by users.
- Reduced MSIT resources required for configuration and implementation, through increased automation. The automation tools and capabilities enabled through the FIMAutomation, Windows PowerShell cmdlets, and Windows PowerShell 2.0 modules significantly streamline deployment and help with desired customizations.
MSIT and the FIM Product Group designed, implemented, and deployed Forefront Identity Manager 2010 R2 not only to improve on the product's existing features, but also to enable FIM to deliver the scale and flexibility required to accommodate the evolving identity-management needs of Microsoft.
The team focused on improving synchronization to enable FIM to meet scale and performance targets, which included reducing server utilization, and incorporating more flexible configuration abilities to meet individual departments' needs. The upgrade also enabled users to set custom UI parameters and utilize new self-management tools.
By largely automating the deployment process and using Windows PowerShell cmdlets, MSIT was able to deploy FIM R2 relatively seamlessly, without significantly affecting enterprise-wide operations.
Resulting benefits have included a 100 percent increase in object server capacity, and a 6-fold increase in object export synchronization rate. Finally, by increasing automation, MSIT also substantially streamlined the entire deployment process.
The following links point to documents that this case study references, or to information that expands on this case study's content:
- Microsoft Technical White Paper
Enhancing and Simplifying Identity Management
- Microsoft Business Case Study
Microsoft IT Reduces Identity Management Costs
- Microsoft TechNet articles
- Microsoft TechNet videos
Dev Center: Custom Activities and Workflows
For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Order Center at (800) 933-4750. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information via the World Wide Web, go to:
© 2012 Microsoft Corporation. All rights reserved.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Active Directory Domain Services, Microsoft Forefront Identity Manager 2010 R2, Microsoft SharePoint Services, Windows, Windows PowerShell, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.