Site Server - Site Server 3.0 P and M Scaling Summary

August 1999 

Personalization & Membership Scaling Summary

This document provides an overview of the effectiveness with which the Membership Directory can scale. Details are provided in a separate document included in the Microsoft® Commercial Internet System (MCIS) Resource Kit titled Site Server 3.0 Membership Directory Capacity and Performance Analysis. 

In this summary, several assumptions are made about a representative user profile. Scalability is discussed based on these assumptions, and on the observed costs of the transactions in the Membership Directory. The two factors that must scale are LDAP Servers (computers that run instances of the LDAP Service) and the SQL Server disk subsystem.

Operational and database design and directory organization are not taken into consideration in this summary. This is purely a performance and scaling view of the Membership Directory.

Note Your performance numbers may vary from those provided here based on your hardware and software platform.

The following system configurations were used in the original test scenarios:

1 One LDAP server was used to test LDAP processor scaling. Three LDAP servers were used to test scaling to multiple servers.

2 To test Membership Directory database scalability, the database was partitioned across two Microsoft SQL Servers.

3 Microsoft SQL Server was configured with 75 percent available RAM, 300 user connections, no truncate on checkpoint, and 120 minute checkpoint interval.

The following chart and table illustrate the number of servers required to support between five and 50 million members. These assumptions are made:

• At peak, concurrent users are two percent of total members. 

• Users need to fetch data from the Membership Directory every five minutes. 

• The data requested from the Membership Directory is used to satisfy personalization and authentication requests. Queries for e-mail name resolution are not included in this profile. 

• All servers are four processor Pentium Pro200 MHz⁴. The Membership Directory is equipped with high-end disk subsystems including four controllers per server, two cabinets per controller and seven disks per cabinet. The disk subsystem is configured to be RAID 1. 

⁴ The LDAP Servers are expected to process two times as many transactions with the MDAC 2.0 Drivers. This will reduce the number of LDAP servers by one half.

Chart 1 Number of LDAP and SQL Servers Required vs. Total Members 

The following is a table representation of the information contained in Chart 1.

Table 1 Number of LDAP and SQL Servers Required vs. Total Members 

The requirements for LDAP Servers are due to CPU. The requirements for SQL servers are due primarily to the disk subsystem throughput capacity, followed closely by disk space requirements.

Calculation Details

This section offers details of the calculation presented in the preceding chart and figure. It includes a configuration of a typical large-scale Personalization & Membership (P&M) implementation which, as indicated above, does not take into account partitioning and limiting database sizes based on operational needs.

Also included is a discussion of the transactions used in P&M and the cost of each of these transactions in terms of CPU and Disk, where applicable. Sample calculations are provided for Chart 1 and Table 1, as well.

Personalization & Membership Configuration Overview

The following diagram depicts the way P&M would be rolled out. The key here is a tiered approach, with the Web servers up front, supported by the proper number of LDAP Servers, supported in turn by the correct number of SQL Servers, and database partitions.

Diagram 1 Example of Personalization & Membership (P&M) Configuration 

User Profile

The following example highlights factors that go into the user profile and how they affect membership scaling. The main scaling factor is the number of trips a client makes to the backend SQL Server.

A member signs on to the site and makes a request to a secured or personalized page.

The Authentication Service on the Web server has no knowledge of the user, so it makes a call to the LDAP Server to retrieve user information. The LDAP Server performs a lookup in the database and retrieves the user's authentication, authorization and personalized profile. This information is passed back to the Authentication Service on the Web server. The Authentication Service caches this information. From now on, when the user requires authentication, authorization, or personalization information, the Authentication Service cache will service the request from cache as long as the user maintains a session with the particular Web server, or until the user's information in the Membership Directory changes or is updated. Note that some installations may not use caches at all for security (SetInfo clears cache).

The main factor in P&M is the number of trips a user makes to the LDAP and SQL Servers. In theory, a user will connect to one server and only make one trip to the backend servers, and satisfy the rest of the queries from the Authentication Service cache. In reality, however, the user does not stay connected to the same server. Therefore, the number of unique Web servers the user connects to is very important for P&M.

Session duration is assumed to be 30 minutes, with a five-minute sticky period to a particular Web server. Therefore, each user will make six trips to the backend servers, regardless of other user behavior.

If a user makes six trips to the server in 30 minutes, then that user will make .003333 queries per second. These queries are estimated to be 80 percent base level searches, 10 percent one level searches and 10 percent sub tree searches.

It is estimated that 20 percent of these queries write to the Membership Directory. These are personalized pages where a property is modified or added and has to be written back to the Membership Directory. An estimated 80 percent of the requests will modify an existing property, and 20 percent will add a property.

Thus, for purposes of this summary, it is assumed that:

• A representative user will make 0.003333 read queries per second, with 80 percent base level, 10 percent one level, and 10 percent sub tree. 

• In addition to read queries, a user will issue writes to the Membership Directory at the rate of .0000667 per second, with 80 percent resulting in modification of an existing attribute, and 20 percent, adding an attribute. 

• New users will sign up for the site. The new user growth rate is estimated at one percent per month. This user population will generate add records. The number of adds per second will depend on the number of current members. 

The foregoing data was used to calculate the data presented in Chart 1 and Table 1.

Having defined the user profile, it is next necessary to address the cost of each of the transactions. The following section discusses the cost of each query at the LDAP Server and the SQL Server.

Transaction Costs and Calculations

Cost per transaction was calculated by exercising each of the transactions separately and watching their effect on resources. Various loads were used to measure the cost of each transaction. For verification purposes, all transactions were run concurrently, and the calculated and observed resource utilization was graphed for various user loads. Observed and calculated resource utilization were within 15 percent of each other.

Because of the behavior of the LDAP Server, the costs are represented by the following equation: R x (C1+C2xR+C3xR^2).

With C1, C2, and C3 given in the table below under cost, R is the rate of the transaction given under the column Rate Range. The equation holds true for all ranges of rate from one to maximum.

Table 2 Operation Costs 

5 Cost expressed in Pentium Pro Equivalent MHz (PPEM). This is total MHz on a Pentium Pro server. For example a 4 x 200 Processor has 800 PPEMHz. Transaction cost is calculated by taking %CPU times total PPEMHz on the server divided by Trans/sec.

6 Limiting factor was the disk subsystem in the test server. 6 disk RAID 0.for data.

From the above costs, the number of PPEM for Base Object Lookup can be calculated for 10 million users as follows:

10 million users x 2 percent concurrent = 200,000200,000 concurrent x 0.003333 queries per user per second = 666.6 queries/second666.6 x 80 percent base object lookup = 533.28

As shown in Table 1, the rate range has a maximum of 180. Dividing 533.28 by 180, you get 2.96. Rounding up, you get 3. Thus you will need to support a transaction rate of 178 on each server. 533/3 = 178.

To solve for CPU cost, use R x (C1+C2xr+C3xr2). From Table 1 you get C1= 2.6782, and C2 = 0.0048, and C3 = 0. Therefore:

178 x (2.6782 + .0048x178) = 628 PPEM

Similarly you can calculate for the rest of the profile to obtain the results in Chart 1 and Table 1.

Using the SQL Server cost table, you can calculate for CPU and disk utilization. SQL Server CPU costs are given in PPEM. The Disk costs are given in number of operations (writes and reads) per second.

SQL Server Costs for Non-Cached Transactions 

Using the 10 million example, you have the following:

10 million users x 2 percent concurrent = 200,000200,000 concurrent x 0.003333 queries per user per second = 666.6 queries/second666.6 x 80 percent base object lookup = 533.28533 x .26 = 138.6 PPEMhz.

This configuration assumes a worst case scenario, that none of the accounts being looked up is cached in the SQL Server cache.

CPU is not the bottleneck for SQL Server, so you will calculate the number of disks required for throughput. Again, take the 10 million-user scenario.

10 million user x 2 percent concurrent = 200,000200,000 concurrent x 0.003333 queries per user per second = 666.6 queries/second666.6 x 80 percent base object lookup = 533.28

You require 4 reads for each lookup, so the total number of reads is:

4 x 533.28 = 2133.12

A disk can do about 50 random reads per second, so this will translate into 44 RAID 1 disks. Since a cabinet can only hold 7 disks and only 2 cabinets can be hooked to one controller, this would translate into 8 cabinets with 7 disks each and 4 controllers. One server with a very high-end disk subsystem would support this.

As demonstrated, the number of SQL Servers is dependent on the disk throughput and not on CPU utilization.