Tuning Windows NT Server Disk Subsystems

By Curt Aubley

Tips for keeping your system in tip-top shape

Article from Windows NT Magazine

Performance! Everyone wants Windows NT servers and workstations to run as fast as possible. Keeping your disk subsystem running at its best is an important step in improving the overall performance of your NT solution. In my article, "Sizing Your NT RAID Array," August 1998, I showed you how to determine when your NT disk subsystem becomes a bottleneck and how to size additional disk capacity for your NT system. But how do you get the most from an additional standalone hard disk or RAID array investment?

In this article, I'll show you how to maximize the performance of your extra disk capacity when implementing NT disk subsystems and RAID arrays, regardless of any vendor-specific hardware tweaks. You can use NT's built-in tools and a freeware tool to quickly optimize your disk subsystem (hardware- or software-based RAID). To accomplish this task, you must understand the performance characteristics of the standalone disk or RAID technology you are working with and the workload characteristics of your existing disk subsystem. Using this information, you can load balance your disk subsystem and tune the disk allocation unit size. Finally, because your performance increase can vary according to your computing environment, I'll show you how I tuned and tested my disk subsystem.

Common RAID Characteristics

Most systems administrators add disk subsystem capacity in the form of RAID arrays. When you use NT's internal software RAID technology or hardware-based vendor RAID technology, RAID 0, 1, and 5 are the most common RAID levels for NT Server environments. (For information about RAID level technology, see Table 1.)

TABLE 1 Relative RAID Performance Guide

Disk I/O Characteristics (1st is fastest, 2nd is second fastest, 3rd is slowest)

When you select RAID configurations, you need to consider many important factors, such as cost and availability level required, in addition to just performance; however, this article focuses on performance. For information about other factors to consider, see the sidebar, "What to Look for When Selecting a Scalable RAID Array."

Regardless of the RAID level you select, use a hardware-based RAID solution if you implement any NT solution that supports more than three disks and requires any level of high availability and performance. Hardware-based RAID adapters and controllers provide much better performance than NT's built-in software-based solution provides and are particularly valuable when you implement RAID 5. With RAID 5, hardware-based RAID adapters offload parity processing and some of the associated interrupt traffic, unlike software-based RAID. This offloading results in improved overall NT performance.

When you're selecting the appropriate RAID level, consider the relative performance characteristics that each RAID level provides. Table 1 illustrates the relative performance level of the various RAID levels in several I/O workload environments and can help you match the appropriate performance to your system's disk I/O characteristics.

Grouping Similar Disk Activities

Configuring one large RAID 5 array to handle all your disk I/O needs might appear to be the easiest solution, but this approach is not always a wise decision. You can dramatically improve performance by matching the performance characteristics of each RAID level with your disk workload patterns. For example, a Microsoft Exchange Server environment contains both sequentially write-intensive log files and random information-store data files. Instead of using one RAID 5 array for both activities, you'll achieve the greatest performance by placing the write-intensive log files on their own RAID 1 array and leaving the random information-store data files on the RAID 5 array. This approach provides better performance because you're moving the write-intensive workload away from the RAID 5 array, which exhibits slower write performance than a RAID 1 array exhibits. Configuring the RAID levels to match your workload patterns improves the response times of the disk subsystem and ultimately the NT system.

If you use a stress-testing tool to measure your server's performance, you can quantify the overall server performance benefit of using multiple RAID levels or adding extra standalone disks. If you don't have a stress-testing tool available, you can use two counters (Avg. Disk sec/Write and Avg. Disk sec/Read) under the LogicalDisk object in NT's Performance Monitor to help you determine whether using multiple RAID levels or standalone disks will increase performance. The Avg. Disk sec/Write counter measures the average time in seconds to write data to the disk, and the Avg. Disk sec/Read counter measures the average time in seconds to read data from the disk. Look at these two counters before and after you change your disk subsystem configuration. If the workload on your server is roughly the same before and after you make changes to your disk subsystem, you will see significant improvements in these two metrics after you implement multiple RAID levels or add standalone disks. Remember, these values will always be zero if you haven't run Diskperf -ye from the NT command prompt and rebooted your NT system to activate NT's collection of disk metrics. Numerous other techniques and Performance Monitor counters can help you measure increased application performance after you have made your changes to your disk subsystem. For additional techniques, see "Related Articles in Windows NT Magazine," page 76.

Disk Workload Characteristics

How can you determine what type of disk workload your server is experiencing so that you can appropriately distribute the disk activities across multiple RAID levels or standalone disks? Performance Monitor provides two counters (% Disk Read Time and % Disk Write Time) under the LogicalDisk object that let you identify disk subsystem workload characteristics. The % Disk Read Time counter measures the percentage of elapsed time that the selected disk is busy servicing read requests, and the % Disk Write Time counter measures the percentage of elapsed time that the selected disk is busy servicing write requests.

Using these counters, you can determine how much time your disk spends writing and reading data. These counters provide a high-level view of the type of disk activity you must plan for. Use this information with the information in Table 1 to select the RAID levels that provide the best performance for your environment. For example, if the value for % Disk Read Time is 10 percent and the value for % Disk Write Time is 90 percent, consider either RAID 1, RAID 0, or a standalone disk (for the latter option, remember that you forfeit any disk fault tolerance for improved performance). As Table 1 shows, RAID 5 write performance is lower than other RAID levels and standalone disks because every RAID 5 write incurs four disk operations: read data, read parity data (compare the two using the CPU), write the data, write the parity data. Conversely, if the value for % Disk Read Time is 80 percent and the value for % Disk Write Time is 20 percent, RAID 5 is a good choice.

Many enterprise networks have a mixed read and write environment with some RAID devices experiencing much higher workloads than others. In these instances, you need to load balance your disk devices for optimal performance.

Load Balancing Your Disks

Throwing more hardware at a bottleneck in the disk subsystem isn't always effective. By load balancing your existing disk subsystem, you can capitalize on your investment and better understand when and where to add disk hardware, if needed. The key to load balancing your NT Server disk subsystem is to identify which disk device is the bottleneck, understand the characteristics of the application using the disk device, and determine which files on the disk device the application is using. Performance Monitor is an excellent tool to help you identify which disk device is the bottleneck. (For information about using Performance Monitor to improve disk I/O, see "The Beginner's Guide to Optimizing Windows NT Server, Part 2," August 1997.) Unfortunately, Performance Monitor alone isn't enough. To understand how and where to distribute your disk workload, you must know which applications (processes) access which files on the disk device.

After you identify which disk device is the bottleneck, you can use two techniques to help you isolate the files on the disk device your applications use. The first technique involves using NT's internal auditing feature (available under Administrative Tools/UserManager/Policies/Audit) to alert you when applications access specific files. This technique is helpful, but the auditing feature generates and sends output to the Event Monitor and can be challenging to decrypt. Auditing also adds a lot of overhead to both the CPU and the disk subsystem selected for auditing.

The second technique, which I find easier to implement, is using Systems Internals' freeware tool, Filemon.exe, which is available at https://www.sysinternals.com . After you download this utility, you just unzip the file and run the utility during high usage periods to get a feel for the specific files your applications are accessing on the disk subsystem, as Screen 1 shows.

Screen 1

Filemon gives you the data you need to load balance your disk subsystem. As Screen 1 shows, you can determine which process is accessing which part of the disk subsystem. By using Filemon with Performance Monitor, you can decide which disk devices to move files from (disks with a high % Disk Time--a combination of read time and write time), which files to move, and where to move them (disk devices whose % Disk Time is low).

Not only does Filemon tell you which applications access which files on which disk devices, but it can also show you whether the disk requests are a read or write activity. As a result, you have more granular data to use when deciding which RAID level to use or when adding standalone disks.

Be careful when you move files around to different volumes under NT. Make sure that the shares and permissions are properly set after you move the files. If the files are associated with a specific application, you might need to move the files using the application or update the Registry. You can determine how you move the files in the disk subsystem according to the applications running in your environment. Regardless of your environment, you will want to have a complete system backup available to restore from if necessary. I also suggest that you make sure no active users are on your system when you make changes to your disk subsystem; otherwise, frustrated end users might complain. Always remember to test your application before and after you move any files to ensure you don't accidentally break anything.

Tuning the Allocation Unit Size

NTFS uses clusters as the fundamental unit of disk allocation. A cluster consists of a fixed number of disk sectors. When you use the Format command or NT's Disk Administrator, clusters are known as the allocation units. In NTFS, the default allocation unit size depends on the volume size. Using the Format command from the command line to format your NTFS volume, you can specify a variety of allocation unit sizes for a specific NT disk volume. For example, to format a volume using an 8KB allocation unit size, go to the command prompt and type

Format k: /f:NTFS /A:8192

For information about the Format command, go to the command prompt and type

Format /? | more

The default allocation unit size that NT provides is a great place to start if you're unaware of the disk workload characteristics. Before you set up a RAID array or new standalone disks, you need to determine the size of the average disk transfer on your disk subsystem and set the allocation unit size to match it as closely as possible. By matching the allocation unit size with the amount of data that you typically transfer to and from the disk, you'll incur lower disk subsystem overhead and gain better overall performance. To determine the size of your average disk transfer, use Performance Monitor to review two counters (Avg. Disk Bytes/Read and Avg. Disk Bytes/Write) under the LogicalDisk object. The Avg. Disk Bytes/Read counter measures the average number of bytes transferred from the disk during read operations, and the Avg. Disk Bytes/Write counter measures the average number of bytes transferred to the disk during write operations.

After you measure the number of bytes written to and read from your disk subsystem, you can set the allocation unit size so that you can achieve maximum performance. Of course, if you want to change the allocation unit size for a file system on the disk device, you have to back up the data from the disk device, use the new allocation unit size to format the disk device, and restore your data. Just as you need to use multiple RAID levels or standalone disks to accommodate different disk workloads, you will want to use this same approach when formatting multiple RAID arrays and standalone disks. Customize each disk device with the allocation unit size appropriate for the projected workload characteristics. If you can't determine the disk workload characteristics, use a smaller allocation unit size on devices that you expect to be random or read-intensive and use a larger allocation unit size for disk devices that will experience sequential and write-intensive workloads.

Your Results May Vary

Just how much your disk subsystem performance will improve using the tuning techniques I've described in this article depends on your environment. To test the differences in matching the allocation unit size, I ran a stress test. My test system consisted of an NT Server 4.0 file server configured with a 100Base-T network card, a 200MHz Pentium Pro processor, 64MB of RAM, a SCSI-based hardware RAID array controller with 16MB of onboard cache, a hard disk for NT, a hard disk for the paging file, and a three-disk RAID 5 array. I tested this system by using the Dynameasure file-server stress-testing tool from Bluecurve operating from four 400MHz Pentium II processor clients with 128MB of RAM and 100Base-TX networking cards. While running the baseline test, I used Performance Monitor to obtain the disk characteristics of the RAID 5 array. I used this information to reformat the RAID 5 array from the default allocation unit size of 4KB to an allocation unit size of 64KB according to my Performance Monitor results. When I retested the system with the same workload, the results were dramatic, as Figure 1 shows. In this chart, higher values are better. As you can see, once the load increased on the file server, the performance of the tuned RAID array improved by more than 5Mbps in some instances.

Figure 1:

Know Your Environment

Although this article presents general performance recommendations, you need to recognize that you'll achieve the best RAID or standalone disk performance improvements when you understand what is occurring on your particular NT system. Then you can make informed decisions when you customize your NT system to match your specific needs.

About the Author

Curt Aubley curt.aubley@washingtondc.ncr.com has worked in the open systems environment for 11 years and specializes in OSs (Windows NT and UNIX), Internetwork architectures, and performance. He is the chief technology officer for OAO and is an MCSE. He is the author of Tuning and Sizing NT Server (Prentice Hall).

The above article is courtesy of Windows NT Magazine.

We at Microsoft Corporation hope that the information in this work is valuable to you. Your use of the information contained in this work, however, is at your sole risk. All information in this work is provided "as -is", without any warranty, whether express or implied, of its accuracy, completeness, fitness for a particular purpose, title or non-infringement, and none of the third-party products or information mentioned in the work are authored, recommended, supported or guaranteed by Microsoft Corporation. Microsoft Corporation shall not be liable for any damages you may sustain by using this information, whether direct, indirect, special, incidental or consequential, even if it has been advised of the possibility of such damages. All prices for products mentioned in this document are subject to change without notice.

International rights.