MIIS 2003 Capacity Planning Test Summary - Database Size

Updated: June 7, 2006

Applies To: Windows Server 2003 with SP1

Previous Sections in This Guide

This section discusses data that was gathered during other performance tests and analyzed to determine any relationship between MIIS 2003 database size and the number of objects being managed.

Test Description

While the other tests presented in this capacity planning document were being run, database size was tracked to see how it was affected by the operations being performed. The information presented in this section is an analysis of that data rather than the result of tests performed specifically to generate size-related information.

General Observations and Recommendations

There is no definitive methodology to determine how the MIIS 2003 database will grow during normal operations. Providing estimates that apply to the broad range of MIIS 2003 deployments is difficult. Factors that impact the database size are:

  • Number of management agents in the solution: More management agents require more storage for the connector space. The connector space stores a complete subset of the information staged from the connected directory.

  • Number of objects in each management agent.

  • The number of attributes being staged for each object.

  • The size of each attribute.

  • The number of multivalued attributes and the size of their values.

  • The schedule at which run profiles are executed in the solution: More frequent scheduling of MIIS 2003 run profiles requires more run history data to be collected.

  • The frequency with which an organization clears the run history. Run history data is collected over time and if the data is not periodically cleared it can drastically increase database size.

  • Maintaining sufficient storage space to be able to clear the run history: MIIS 2003 cleans out the run history information within the MIIS 2003 database within one SQL transaction. The result is that the SQL database log expands in size to contain all the deleted run history information. It does not shrink until the transaction is committed. Thus, you need sufficient additional storage for the database for this expansion. The amount of expansion depends on the size of the MIIS 2003 solution and the amount of run history the organization maintains.

The following section provides examples of the database sizes that where observed during the tests of other MIIS 2003 capacity planning components. Some of these examples can be used to estimate the size of an MIIS 2003 database.

It is recommended that you perform tests in a controlled environment by using a representative sample of data from your production environment to help you estimate your own database size requirements. In doing so, your estimates will be based on actual data and the same type of staging and synchronization operations that will be used in your production environment. Based on these data points, some broad estimates could be made about the projected database size.

Test Scenario

The data presented in this section was gathered from the other tests described elsewhere in this document (specifically, the processor and memory tests). The data was collected from the various platforms used for those tests. For information about the particular server configuration used in each test, see "Server Hardware Configuration" and "MIIS 2003 Configuration" in "MIIS 2003 Capacity Planning Test Summary - Processor" and "MIIS 2003 Capacity Planning Test Summary - Memory" in this document.

Test Results

Data that pertains to the database size from two test scenarios is presented here. The first set of data is from tests that determined the processor requirements for MIIS 2003. This scenario consisted of two management agents that imported data into the metaverse and then exported information to a third management agent. During these tests, the database growth was measured.

The second set of data was gathered from the memory tests. It provides an example of what happens to the database size when the number of management agents that process data increases. That scenario used ten management agents to import and synchronize data into an MIIS 2003 database. Database growth was also monitored during these tests.

Collectively, the data provided below indicates both total size and the rate the database grew.

Database Sizing Based on Processor Testing Scenario

For a complete description of the server configuration used to generate this data, see the "Server Hardware Configuration" and "MIIS 2003 Configuration" information in "MIIS 2003 Capacity Planning Test Summary - Processor" in this document.

Table 25: Physical Database File Size Growth (File level measured in MB)

Management Agent Operation 10,000 Objects 50,000 Objects 100,000 Objects 200,000 Objects 500,000 Objects

Empty Database

4

4

4

4

4

Active Directory

Full Import

4

4

4

4

4

TXT

Full Import

40

169

329

641

1,662

SQL

Full Import

71

329

641

1,374

3,564

Active Directory

Full Sync

71

329

641

1,374

3,564

TXT

Full Sync

139

705

1,374

2,945

7,640

SQL

Full Sync

169

853

1,662

3,240

8,404

Active Directory

Export

169

853

1,662

3,240

8,404

Active Directory

Delta Import

186

853

1,829

3,564

8,404

SQL

Delta Import (20% changed records)

186

853

1,829

3,564

8,404

SQL

Delta Sync

186

853

1,829

3,564

8,404

Active Directory

Export

186

853

1,829

3,564

9,245

Active Directory

Delta Import

186

853

1,829

3,564

9,245

SQL

Delta Import (10% deleted records)

186

853

1,829

3,564

9,245

SQL

Delta Sync

186

938

1,829

3,564

9,245

Active Directory

Export

186

938

1,829

3,564

9,245

Active Directory

Delta Import

186

938

1,829

3,564

9,245

noteNote
These figures do not include any additional increase in size due to run history data that would normally be generated by run profile activity in a production environment. In the test environment, the run profiles were only processed a limited number of times. In a production environment, they might get processed multiple times daily.

The database sizes are based upon the physical MDF file size on the file system. This is why the file grows to a certain point and then remains constant. This is due to the extents that SQL Server performs. By default this is a 10% growth in database size. In the case of these tests, the data added to the database did not warrant another extent.

Table 26: Description of run profile operations used to generate the data in Table 25.

Management Agent Operation

Active Directory

Empty Active Directory staged to Active Directory management agent connector space

TXT

Variable number of user staged from TXT management agent (see columns in Table 16 above)

SQL

Variable number of user staged from SQL management agent (see columns in Table 16 above)

Active Directory

Synchronization of Active Directory management agent connector space

TXT

Synchronization of TXT management agent connector space

SQL

Synchronization of SQL management agent connector space

Active Directory

Export the number of users to Active Directory

Active Directory

Delta Import from Active Directory management agent

SQL

Delta Import of 20% change in published number of objects

SQL

Delta Synchronization of changed records

Active Directory

Export the changes to Active Directory

Active Directory

Delta Import from Active Directory management agent

SQL

Delta Import of 10% deletions in the published number of objects

SQL

Delta Synchronization of the deleted records

Active Directory

Export the deletions to Active Directory

Active Directory

Delta Import from Active Directory MA

Database Growth Based on Multiple Management Agent Testing

During these tests, ten identical text sources were sequentially imported and synchronized into an MIIS 2003 server. All the data within each of the management agents were identical and thus joined 100% to other records within the metaverse. During these tests, the file size of the database, as reported by SQL Server and the file system, was recorded.

For a complete description of the server configuration used to generate this data, see "Server Hardware Configuration" and "MIIS 2003 Configuration" in the "MIIS 2003 Capacity Planning Test Summary - Memory" in this document.

Chart: Ratio of data growth during staging

Table 27: Ratio of growth during staging, based on database size compared to initial database size. (Used to plot Figure 16)

Objects MA1 MA2 MA3 MA4 MA5 MA6 MA7 MA8 MA9 MA10

10,000

1.00

1.95

2.92

3.83

4.86

5.78

6.68

7.61

8.74

9.67

50,000

1.00

2.00

2.99

4.00

5.03

5.98

6.94

7.98

9.10

10.06

100,000

1.00

1.99

2.99

4.02

5.04

5.99

6.95

8.01

9.11

10.07

200,000

1.00

2.01

3.01

4.04

5.06

6.02

6.97

8.03

9.16

10.12

Observations
  • We can see a linear relation to the growth of the database, which was expected because all the sources that were imported into the database were equal in size.

  • Note that with the run history information and the extra management agent data, the database doubles in size after each run profile staging operation.

Chart: Ratio of data growth during synch

Table 28: Ratio of growth during synchronization, based on database size compared to initial database size. (Used to plot Figure 17)

Objects MA1 MA2 MA3 MA4 MA5 MA6 MA7 MA8 MA9 MA10

10,000

1.00

1.02

1.04

1.05

1.07

1.09

1.11

1.13

1.15

1.16

50,000

1.00

1.02

1.04

1.06

1.08

1.10

1.12

1.14

1.16

1.18

100,000

1.00

1.02

1.04

1.06

1.08

1.11

1.13

1.15

1.17

1.19

200,000

1.00

1.02

1.04

1.06

1.08

1.10

1.13

1.15

1.17

1.19

Observations
  • Again a linear growth of the database is apparent, which was expected because all the sources that were synchronized into the database were equal in size and joined perfectly to all existing data.

We also notice that the cost in terms of database size to facilitate the linking between is quite small. Because all records in all connector spaces were identical, the metaverse itself did not grow after the initial projection. Upon closer examination, we found that most of the information was written to the “mms_csmv_link” table, which contains linkage between the various connector spaces and the metaverse objects. The rest of the information was written to the run history tables in the MIIS 2003 database.

Table 29a: Database growth as reported by both SQL Server and the server file system during processing of 10,000 and 50,000 objects (listed in Kbytes)

MA Operation 10,000 SQL DB MDF DB 50,000 SQL DB MDF DB

Empty Database

904

2,359

904

2,359

TXT01

Import

28,192

33,227

135,728

153,289

TXT02

Import

55,024

59,048

271,632

298,910

TXT03

Import

82,216

86,442

405,424

437,649

TXT04

Import

108,032

115,081

542,664

582,550

TXT05

Import

137,128

153,289

682,360

704,905

TXT06

Import

162,816

185,532

811,736

853,017

TXT07

Import

188,448

204,079

942,608

1,032,192

TXT08

Import

214,624

224,526

1,083,528

1,135,411

TXT09

Import

246,408

271,712

1,234,616

1,373,897

TXT10

Import

272,560

298,910

1,364,992

1,511,326

TXT01

Sync

299,488

328,794

1,504,248

1,662,452

TXT02

Sync

304,400

328,794

1,534,200

1,662,452

TXT03

Sync

310,576

328,794

1,564,328

1,662,452

TXT04

Sync

315,552

328,794

1,595,320

1,662,452

TXT05

Sync

321,720

361,693

1,623,848

1,828,717

TXT06

Sync

325,304

361,693

1,654,912

1,828,717

TXT07

Sync

331,320

361,693

1,687,128

1,828,717

TXT08

Sync

337,160

361,693

1,716,312

1,828,717

TXT09

Sync

343,488

361,693

1,744,944

1,828,717

TXT10

Sync

347,272

361,693

1,773,856

1,828,717

Table 29b: Database growth as reported by both SQL Server and the server file system during processing of 100,000 and 200,000 objects (listed in Kbytes)

MA Operation 100,000 SQL DB MDF DB 200,000 SQL DB MDF DB

Empty Database

904

2,359

904

2,359

TXT01

Import

264,320

298,910

518,864

582,550

TXT02

Import

527,168

582,550

1,042,680

1,135,411

TXT03

Import

791,160

853,017

1,562,704

1,662,452

TXT04

Import

1,061,400

1,135,411

2,093,984

2,212,823

TXT05

Import

1,331,976

1,373,897

2,623,080

2,945,384

TXT06

Import

1,582,112

1,662,452

3,121,848

3,239,969

TXT07

Import

1,836,888

2,011,628

3,615,928

3,920,429

TXT08

Import

2,116,352

2,212,823

4,166,984

4,312,465

TXT09

Import

2,408,968

2,677,604

4,752,512

5,218,173

TXT10

Import

2,661,424

2,945,384

5,253,304

5,740,036

TXT01

Sync

2,940,536

3,239,969

5,821,616

6,314,066

TXT02

Sync

3,002,392

3,239,969

5,940,104

6,314,066

TXT03

Sync

3,062,384

3,239,969

6,061,168

6,314,066

TXT04

Sync

3,126,416

3,239,969

6,184,776

6,314,066

TXT05

Sync

3,185,096

3,563,979

6,301,720

6,945,505

TXT06

Sync

3,249,712

3,563,979

6,424,568

6,945,505

TXT07

Sync

3,315,240

3,563,979

6,551,032

6,945,505

TXT08

Sync

3,375,392

3,563,979

6,669,616

6,945,505

TXT09

Sync

3,431,872

3,563,979

6,787,696

6,945,505

TXT10

Sync

3,492,112

3,920,429

6,906,944

7,640,056

Observations
  • Because of the way that the MDF file gets extended by SQL server, there is usually a huge jump in growth at various points.

Next

See Also

Community Additions

ADD
Show: