Getting the Correct Troubleshooting Tools

When troubleshooting the performance of a server that is running Microsoft® Exchange Server 2003, there are tools that come with the Microsoft Windows® operating system or are available on the Internet that can be of help. The tools listed in the following table are useful when diagnosing or preventing a bottleneck, and helping to isolate the problem.

Performance Tools for Exchange 2003

LoadSim and ESP are tools that are helpful when testing systems to ensure the health of the systems before going into production. You can use the other tools to help diagnose bottlenecks in production servers.

Performance

Performance is a Microsoft Management Console (MMC) snap-in that enables monitoring numerous subsystems and software over time. It provides a common infrastructure for reporting data based upon performance counters. These counters are organized hierarchically, as follows, by object, counter and (optionally) instance:

• Performance Object   This is the part of the computer being monitored. Some of the most commonly used objects are Processor, Memory, and PhysicalDisk. Installing Exchange 2003 adds new objects, such as MSExchangeIS, to the performance object list.

• Counters   The counters available for a performance object are the parts of the object you can monitor. For example, you can monitor the available bytes, kilobytes, and megabytes of memory, as well as the page faults per second or total pages per second, for a Memory object.

• Instances (optional)   There may be multiple objects or counters to monitor on the computer. For example, when looking at counters under the Processor object on a multiple processor computer, you see as many instances as there are processors on that computer. You can choose to monitor only a specific processor or all processors.

There are many performance counters covered in this article. Using the Performance snap-in, you can easily glance at the current values of these counters and verify the current status of each counter. However, possibly the most interesting feature is the ability to log any number of counters, objects, and instances across any number of machines to either a file or an SQL database.

The performance counters referenced in this article have the following format:

Performance Object(Instance)\Counter

Knowing When to Monitor Performance

Knowing which counters to monitor using the Performance snap-in is only part of monitoring performance. You also need to know when to monitor these counters. A server has many states and, by monitoring each of these states, you derive an understanding on what is normal server operation and what is not.

When you bring an Exchange server online, it takes some time until Exchange fills the caches properly and populates all memory structures. This state is called the transient state. The length of the transient state varies based upon the size of the server (number of processors, memory, and other factors) and the load applied to it. Typically, the transient state varies from 2 hours to 1 per day.

After the transient state is over, the server enters a state where it should exhibit a consistent load with a predictable variance. This state is called the stationary state, and should hold indefinitely unless the load changes or external factors take place, such as backups, restores, database maintenance, and so on.

To analyze a server, it is important to log the performance of server with as many data points as possible, so that the sampling rate is faster than the effect being measured. If possible, a cyclical performance degradation should be logged with data points before, during, and after it took place—the entire cycle. In addition, it is recommended to use a sampling resolution of at least one minute or less to capture performance issues. For example, disk spikes that subside in 5 minutes can be tracked with 30-second Performance logs.

If there is no evident degradation in specific periods of time, logging during an extended period such as one entire day cycle in the life of a server is generally very revealing. It is important to understand how the server reacts as users start using the server in the morning, during peaks hours, and at the end of the day. It is also important to understand how the server reacts during database maintenance and backup cycles. Sometimes, an ill-conceived backup schedule may force the database maintenance not to finish properly, which affects the server performance later.

Load Simulator (LoadSim)

Load Simulator or LoadSim 2003 is a tool that simulates the performance load of MAPI clients. It is useful tool for administrators who are sizing servers and validating a deployment plan. Specifically, LoadSim helps you determine if each of your servers can handle the load that you intend them to carry. Another use for LoadSim is to help validate the overall solution.

However, when it comes to sizing servers completely, LoadSim does not account for all factors. LoadSim does not simulate the following factors that can affect your server capacity planning:

• Incoming unsolicited commercial e-mail (also known as spam) from the Internet

• Incoming Simple Mail Transfer Protocol (SMTP) mail flow from the Internet or other sites within your organization

• Use of non-MAPI protocols for account access, such as Post Office Protocol version 3 (POP3) and Internet Message Access Protocol version 4rev1 (IMAP4)

• Use of mobile devices

• Public folder utilization

In addition, LoadSim does not give a complete picture with regards to user experience, and its results should not be interpreted in that aspect.

For more information about LoadSim 2003, read the file, Using LoadSim 2003.doc, that comes with the tool.

Exchange Stress and Performance

You can use Exchange Stress and Performance (ESP) to simulate arbitrarily several client sessions concurrently accessing one or more Exchange 2003 servers.

ESP provides modules that simulate client sessions over the following Internet protocols/APIs:

• WebDAV (for Microsoft Outlook® Web Access)

• IMAP4

• Lightweight Directory Access Protocol (LDAP)

• OLE DB

• Network News Transfer Protocol (NNTP)

• POP3

• SMTP

• Outlook Mobile Access Sync

• Outlook Mobile Access Browse

Therefore, ESP should be considered as a tool that is similar to LoadSim, but is used when validating deployments that use mobility features and Internet protocols that LoadSim does not cover.

For more information about ESP, read the documentation that comes with the tool.

Network Monitor

Network Monitor enables you to detect and troubleshoot problems on LANs. Using Network Monitor, you can:

• Identify network traffic patterns and network problems. For example, you can locate client-to-server connection problems, find a computer that makes a disproportionate number of work requests, and identify unauthorized users on your network.

• Capture frames (packets) directly from the network.

• Display, filter, save, and print the captured frames.

For instructions for about using Network Monitor to troubleshoot performance, see the "Troubleshooting Network Utilization" section later in this article. For more information about Network Monitor, see the following Microsoft Knowledge Base articles:

• Microsoft Knowledge Base article 294818, "Frequently Asked Questions About Network Monitor"

• Microsoft Knowledge Base article 148942, "How to Capture Network Traffic with Network Monitor"

Filemon

Filemon is a third-party tool available from Sysinternals. This tool monitors and displays file system activity on a system in real-time. Its advanced capabilities make it a powerful tool for exploring the way Windows works, seeing how applications use files and DLLs, or assessing problems in system or application file configurations. The time stamping feature of Filemon precisely indicates when every open, read, write, or delete occurs, and its status column indicates the outcome. Filemon begins monitoring when you start it, and its output window can be saved to a file for off-line viewing. It has full search capability and filters.

For more information about the Sysinternals Filemon tool, see Troubleshooting I/O Transfers.