Using Exchange Server 2003 Stress Tools in a Test Lab

  • Article

This article will help you test an enterprise Microsoft® Exchange Server 2003 environment in a non-production test lab before you implement your plan in a production environment. If you are planning to deploy Exchange Server 2003 for the first time, or are planning to upgrade to Exchange Server 2003 from earlier versions of Exchange, it is a good idea to test your deployment plan in an environment that is similar to your production environment. If you are upgrading to new hardware, or to a new storage solution, it is also a good idea to test Exchange in the new environment before implementing the new hardware or new storage devices in your production environment.

By testing in a pre-production environment, you can:

  • Verify that you meet hardware requirements.
  • Plan for a smooth transition for users.
  • Measure the performance improvements that you will gain with the new software or hardware.

What You Will Find in This Article

This article will help you plan, set up, implement, and analyze your tests, using several tools that are available as downloads for Exchange Server 2003. For more information about all Exchange downloads, see Downloads for Exchange 2003 (https://go.microsoft.com/fwlink/?LinkId=25097).

This article includes:

  • Guidelines for tuning each of the stress tools available for your tests, so that you can closely simulate your actual production environment.
  • Explanations and comparisons of the various stress tools, so that you know which ones to use at the different stages of your topology validation and testing.
  • Information about which tools to use together to simulate your production environment.
  • Explanations of how to monitor the tools and the Exchange servers during the tests.
  • Recommendations and guidelines for customizing the tests, if necessary, based on the activity on each of the Exchange servers.

What You Will Not Find in This Article

This article does not provide step-by-step instructions for how to use the tools that are available for running stress tests on Exchange servers. Instead, it provides links to documentation that includes detailed instructions for each tool.

Tools

Use the following tools to test your Exchange Server 2003 deployment:

  • Exchange Server 2003 Jetstress Tool
  • Exchange Server Load Simulator (LoadSim) 2003
  • Exchange Server Stress and Performance (ESP) 2003

Each of these tools verifies a different part of your Exchange Server 2003 deployment plan. By using these three tools together, you can test the underlying storage solution, as well as the overall performance of your servers and storage solution during user activity.

Testing Your Exchange Server 2003 Deployment Plan

There are two stages when stress testing a large Exchange environment in a lab. The first stage involves testing that the storage solution, which can include Storage Area Network (SAN) devices, disk arrays, and internal disks on which Exchange data will be stored, meets your requirements for speed and reliability. The second phase involves testing the entire system, including the Exchange server hardware, network capabilities, storage solution, and client access, while running tests that simulate Exchange user activity.

Testing the Storage Solution

When you test your storage solution, the disk subsystem in the test lab should be as similar to that of the production environment as possible. Use the Jetstress tool to verify that the storage configuration is adequate for the load that Exchange will put on the disks.

Jetstress

Jetstress simulates Exchange disk input/output (I/O) load. It does not simulate actual user activity; rather it simulates the load on the Exchange database files and log files produced by a specified number of users. For more information about how to run Jetstress tests, see Exchange Server 2003 Jetstress Tool (https://go.microsoft.com/fwlink/?LinkId=27883).

When you have completed the Jetstress tests and have validated your disk subsystem, be sure to delete the database files and log files created by the Jetstress tool.

To delete Jetstress database and log files

  1. If you used Jetstress 2004, delete the directory c:\jetstress, and all of its contents.

  2. If you used the command line Jetstress tool, delete the following directories and all of their contents:

    • C:\jetstress
    • G:\Jetstress_db1
    • H:\Jetstress_db2
    • I:\Jetstress_log1
    • J:\Jetstress_log2

User Simulation Testing

User simulation testing involves testing your Exchange environment in a lab using stress tools to validate that the system performs well under load. The two recommended tools for user-simulation stress testing of Exchange servers are LoadSim 2003 and ESP 2003. These tools perform tasks from a client computer in the same way that users perform tasks from an e-mail client. The tasks performed by the tools simulate actions on the Exchange server. Using the LoadSim 2003 and ESP 2003 tools in an environment that is similar to your production environment will help you to validate the user experience in your Exchange organization.

LoadSim 2003

LoadSim 2003 simulates Microsoft Office Outlook® 2003 tasks and sends remote procedure calls (RPCs) to an Exchange server. These RPCs from the client to the server use the MAPI protocol, which accesses the Exchange store directly. For more information about how to run LoadSim, see Exchange Server Load Simulator 2003 (LoadSim) (https://go.microsoft.com/fwlink/?LinkId=27882).

For information about how to make the most of your LoadSim tests, see "Time and Resource Intensive Tasks" later in this article.

User Profiles

There are four default user profiles in LoadSim:

  • Heavy
  • Medium
  • Cached Mode
  • MMB3

The Heavy profile most closely simulates users classified as information workers who use Outlook extensively. Users running the Heavy profile each receive approximately 210 messages per day, and send approximately 50 messages per day.

Users running the Medium profile each receive approximately 140 messages per day, and send approximately 30 messages per day.

The Cached Mode profile simulates users who run Outlook in Cached Exchange Mode. This profile enables the Offline Address Book task and Synchronize Folders task, and disables the Process Inbox and Browse Mail tasks. Users running the Cached Mode profile receive approximately 160 messages per day, and send approximately 45 messages per day.

The MMB3 profile is intended to be used as a benchmark for comparing hardware and software products. It is not intended to simulate a specific type of user in a production environment.

If you have collected user profile data from your production environment, you can use that data to customize your own user profile.

To customize your user profile

  1. Start LoadSim.

  2. On the Configuration menu, click Test Properties.

  3. In the User Groups box, select a user group, and then click Customize Tasks.

  4. On the Tasks tab, edit the tasks in the Tasks list so that they are similar to the profile data you collected from your production environment.

Mailbox Size

The average message size in the default set of messages that is used by LoadSim is approximately 100 kilobytes (KB). You can adjust the initialization parameters in LoadSim to create a topology in which user mailboxes average a particular size, similar to the average mailbox size in your production environment. To keep the mailboxes at approximately the same size throughout the LoadSim test, you can limit the number of messages that LoadSim retains in users' Inboxes and in other folders. By setting these limits to the same number of messages that you set in the initialization, you can keep the user mailboxes at a relatively static size.

To limit the number of messages LoadSim keeps in a user's Inbox and folders

  1. Start LoadSim.

  2. On the Configuration menu, click Test Properties.

  3. In the User Groups box, select a user group, and then click Customize Tasks.

  4. On the Tasks tab, in the Tasks list, select Process Inbox.

  5. In the Maximum Messages in Inbox box, set the Limit to X messages option, where X is the number of messages that will be allowed in the Inbox at a given time.

  6. In the Tasks list, select Browse Mail.

  7. In the Maximum Messages in Folders box, set the Limit to X messages option, where X is the number of messages that will be allowed in all other folders at a given time.

  8. Click OK.

Time and Resource Intensive Tasks

Some activities in the LoadSim profiles have a significant time impact or resource impact on the server. If your Exchange server becomes overloaded, you can adjust tasks in the LoadSim profile to reduce the amount of stress on the Exchange server. The tasks that are described in this section are time intensive or resource intensive, and will cause the greatest stress on the Exchange server.

One task that adds significant stress to the Exchange server is adding dynamic distribution lists to messages.

To add dynamic distribution lists to messages that LoadSim users send

  1. Start LoadSim.

  2. On the Configuration menu, click Test Properties.

  3. In the User Groups box, select a user group for which you want to enable adding dynamic distribution lists, and then click Customize Tasks.

  4. On the Tasks tab, in the Tasks list, select Send Mail.

  5. Select the Add a single DDL to X % of messages sent option, and set the percentage in the box.

  6. Click OK.

Adding a dynamic distribution list to messages adds significant load to the Exchange server, particularly if your dynamic distribution lists are very large. If you want to use dynamic distribution lists, start with dynamic distribution lists that have a small number of aliases in the list, and start by adding a dynamic distribution list to a small percentage of messages. For example, create one dynamic distribution list per mailbox store, and only add a dynamic distribution list to a very small percentage of messages, for example, 1 percent.

To create dynamic distribution lists or change the size of dynamic distribution lists

  1. Start LoadSim.

  2. On the Configuration menu, click Topology Properties.

  3. On the Distribution Lists tab, select the Use dynamic distribution lists check box.

  4. Select either Create one for all LoadSim users, Create one per Server, or Create one per MDB.

  5. Click OK.

  6. On the Run menu, click Create Topology. If you have already created distribution lists and dynamic distribution lists, they will be deleted and re-created with the new settings.

To reduce the load on the Exchange server that sends mail to dynamic distribution lists, designate a separate server as the expansion server for each dynamic distribution list, so that the large dynamic distribution lists will not be expanded on the sending server. For information about how to use expansion servers, see Microsoft Knowledge Base article 328791, "HOW TO: Use Expansion Servers in Exchange 2000" (https://go.microsoft.com/fwlink/?LinkID=3052&kbid=328791).

Another LoadSim task that can add significant stress to the Exchange server is applying random views and sorting orders to the Inbox and other folders. You can configure this task for the Inbox in the Process Inbox task, and for other folders in the Browse Mail task.

Significant stress can also be added to the Exchange server if the parameter for the number of appointments for each user is configured incorrectly. LoadSim creates both single-instance appointments and recurring appointments, based on the percentage of recurring appointments set in the Make Appointments task. All recurring appointments that LoadSim creates have no end date. Therefore, the number of total items in a user’s calendar could be considerably higher than the number indicated in the Initialization tab, if some percentage of appointments is recurring. Calendar tasks can be both time and resource intensive if the calendar is very large.

Logging on users can be time intensive. If you choose to have users log on immediately at the very beginning of the test, all users will log on as quickly as possible, and no other tasks will begin on a client computer until all users from that client are logged on. Logging on 1,000 users, for example, should take fewer than ten minutes. If you clear the Log on immediately at the very beginning of the test check box, as soon as a user has logged on, LoadSim will begin performing other tasks for that user. In this situation, it could take an hour or more for 1,000 users to log on. Although logging on users immediately at the beginning of the test is faster, it does not simulate the way users in a production environment log on. In a production environment, they log on at different times, and start performing tasks as soon as they log on.

To configure the way users log on

  1. Start LoadSim.

  2. On the Configuration menu, click Test Properties.

  3. In the User Groups box, select a user group for which you want to configure logons, and then click Customize Tasks.

  4. On the Test/Logon tab, either clear or set the Log on immediately at the very beginning of the test check box.

  5. Click OK.

Troubleshooting

To troubleshoot common problems and errors with LoadSim, see the "Troubleshooting and Q & A" section of the Load Simulator 2003 document, which is downloaded with the LoadSim 2003 tool.

Exchange Server Stress and Performance 2003

ESP 2003 can simulate the following protocols:

  • HTTP/DAV (also known as WebDAV)
  • Internet Message Access Protocol version 4rev1 (IMAP4)
  • Lightweight Directory Access Protocol (LDAP)
  • OLEDB
  • Post Office Protocol version 3 (POP3)
  • Simple Mail Transfer Protocol (SMTP)
  • Network News Transfer Protocol (NNTP)
  • Outlook Mobile Access Browse
  • MobileSync (also known as Exchange Server ActiveSync)

For more information about Exchange Server Stress and Performance 2003, see Exchange Server Stress and Performance 2003 ** (https://go.microsoft.com/fwlink/?LinkId=27881).

Customizing Your Protocol Stress Tests

The types of protocol stress you should use in your tests will depend on what combination of client software you anticipate having in your production environment. If your users will be accessing their mailboxes from IMAP4 clients or POP3 clients, then you should include those protocols in the test scenario. If your users will connect to the Exchange server using Outlook, focus your test scenario on LoadSim testing, which simulates Outlook calls to the server. If your users are able to access their mail remotely using Microsoft Outlook Web Access, Outlook Mobile Access Browse, or MobileSync, you should add these protocols to your test. The amount of stress from these mobile clients that you put on the server in your test should be based on the anticipated usage in production. This may change over time, so be sure to test scenarios in which the mobile access is scaled up to account for increases in usage.

Example Multiprotocol Test

The following is an example test scenario for a multiprotocol test lab. The Exchange server in this scenario has 1,000 users who use Outlook heavily for mail and calendaring. These same users occasionally access their mailboxes using Outlook Web Access. Approximately 10 percent of the users in this example also have mobile devices and use Outlook Mobile Access Browse and Sync on occasion. Some users in this scenario also access their mailboxes using POP3 clients, and some access their mailboxes using IMAP4 clients.

The test should include 1,000 Heavy LoadSim users. The test should also include the following ESP modules: DAV, Outlook Mobile Access Browse, MobileSync, POP3, and IMAP4. These run while LoadSim stress is running.

The number of concurrent DAV instances in the test should be approximately 100 per Exchange server. This number of DAV instances reflects the fact that, in this example, users usually connect to their mailboxes with Outlook, and use Outlook Web Access as a secondary client.

The number of concurrent Outlook Mobile Access Browse instances and MobileSync instances for the test should be approximately 30 each. This number of Outlook Mobile Access Browse and MobileSync instances reflects the fact that, in this example, only approximately 10 percent of the users in the organization have mobile devices. Of those users, fewer than half are connected at the same time.

Before you determine the number of concurrent POP3 instances, it is important to understand that POP3 is not a connected protocol. This means that users are not continually connected to the server. A POP3 client connects to the server, performs a few tasks, and then disconnects. At any given time, the number of concurrent POP3 connections to the Exchange server will be quite low. Therefore, the number of POP3 instances for this test should be five.

IMAP4 is a connected protocol. Unlike POP3, IMAP4 clients remain connected to the Exchange server even during idle periods. Therefore, the number of concurrent connections reflects the anticipated IMAP4 usage. The number of concurrent IMAP4 instances for this test should be 100.

Tips for Running the User Simulation Tests

To successfully run a test of this magnitude with several different protocols, scale up from much lighter stress to this heavy multiprotocol scenario. First, test each protocol individually, to be sure that the configuration and your assumptions about the amount of stress are consistent with the hardware and storage configuration. For protocols that will have a large number of concurrent connections, start with a relatively small number, approximately 10-20 percent of the total number of connections or users. Allow the stress to run for two hours so that the activity on the server has a chance to level out after the initial load from users logging on. After the servers are running smoothly, increase the number of connections in equal increments. Continue increasing the number of connections until you reach the total number of connections for the test. After each protocol has been tested in this way, begin adding multiple protocol stress at the same time, again gradually increasing the number of connections in consistent increments.

While you are scaling up, be sure to look for errors in the tools and in the Event Viewer. Monitor the loadsim.out file on the client to check for errors. Monitor the LoadSim output window to check for queues of pending tasks. If the Tasks Pending counter increases continually, the Exchange server is not able to handle the load being generated by LoadSim. You must investigate the cause of this problem before starting the actual test with all the protocols. If you find errors in the loadsim.out file, see "Troubleshooting and Q&A" in Exchange Server Load Simulator 2003 (https://go.microsoft.com/fwlink/?LinkId=27882). When you are running ESP, be sure to keep a log file, and examine it for connectivity errors, errors in the script or additional files, or ESP configuration errors.

To keep a log file in Exchange Server Stress and Performance (ESP)

  1. Start ESP.

  2. Right-click the host, and then click Properties.

  3. On the Options tab, double-click the value field next to the Log Results parameter, and type Yes in the field.

  4. If you want to archive the log files each time you start an ESP module, double-click the value field next to the Archive Log parameter, and then type Yes in the field.

  5. Check that the value for Log File Path is correct.

  6. Click OK.

The following are tips for troubleshooting ESP errors:

  1. Try running through the entire script with a single instance and no sleep commands in the script. If that is successful, then gradually add more instances until you reach the appropriate number of instances.
  2. Try manually performing the action that is causing errors in ESP. For example, if you have an error with the DAV module, log on to a user’s mailbox by opening a Web browser and going to the Web address [Exchange server]/exchange/[username]. If the problem is with the POP3 module, open a TCP connection to port 110 and try to log on and complete the action that is causing errors. Manual tests often help you to see where the problem is occurring.

Finally, during the scale-up tests and during the actual tests, keep Performance Monitor logs on the Exchange server. For information about which counters to log and which thresholds to be aware of, see Troubleshooting Exchange Server 2003 Performance (https://go.microsoft.com/fwlink/?LinkId=22811).