Overview of Storage Technologies
Topic Last Modified: 2006-05-08
To increase the availability of your Exchange 2003 organization, your Exchange back-end storage solution must be supported by a redundant storage subsystem. When planning your storage solution, familiarize yourself with the following storage-related technologies:
RAID levels Disk array implementations that offer varying levels of performance and fault tolerance.
SAN solutions Storage that provides centralized data storage by means of a high-speed network.
Network-attached storage solutions Storage that connects directly to servers through existing network connections.
Replication technologies Solutions that use synchronous and asynchronous data replication technologies to replicate data within a site (using SANs and LANs) or to a separate site (using virtual LANs).
SAN and network-attached storage solutions usually incorporate RAID technologies. You can configure the disks on the storage device to use a RAID level that is appropriate for your performance and fault tolerance needs. Use the information in the following sections to compare and contrast these storage technologies.
|It is generally recommended that you use a direct access storage device (DASD) or a SAN-attached disk storage solution because these configurations optimize performance and reliability for Exchange 2003. Microsoft does not support using network-attached storage solutions unless they meet specific Windows requirements.|
|For information about SAN and network-attached storage solutions, see Microsoft Knowledge Base article 328879, "Using Exchange Server with Storage Attached Network and network-attached storage devices."|
Before deploying a storage solution for your Exchange 2003 databases, obtain verification from your vendor that the end-to-end storage solution is designed for Exchange 2003. Many vendors have best practices guides for Exchange.
By using a RAID solution, you can increase the fault tolerance of your Exchange organization. In a RAID configuration, part of the physical storage capacity contains redundant information about data stored on the hard disks. The redundant information is either parity information (in the case of a RAID-5 volume) or a complete, separate copy of the data (in the case of a mirrored volume). If one of the disks or the access path fails, or if a sector on the disk cannot be read, the redundant information enables data regeneration.
|You can implement RAID solutions either on the host system (software RAID), or on the external storage array (hardware RAID). In general, both solutions provide similar reliability benefits. However, software RAID increases the CPU processing load on the host server. This section assumes that you use a hardware RAID solution rather than a software RAID solution. For information about using software RAID with Microsoft Windows Server™ 2003, see Windows Server 2003 Help.|
To ensure that your Exchange servers continue to function properly in the event of a single disk failure, you can use disk mirroring or disk striping with parity on your hard disks. With disk mirroring and disk striping with parity, you can create redundant data for the data on your hard disks.
Although disk mirroring creates duplicate volumes that can continue functioning if a disk in one of the mirrors fails, disk mirroring does not prevent damaged files (or other file errors) from being written to both mirrors. For this reason, do not use disk mirroring as a substitute for keeping current backups of important data on your servers.
|When using redundancy techniques such as parity, you forfeit some hard disk I/O performance for fault tolerance.|
Because transaction log files and database files are critical to the operation of an Exchange server, you should keep the transaction log files and database files of your Exchange storage group on separate physical drives. You can also use disk mirroring or disk striping with parity to prevent the loss of a single physical hard disk from causing a portion of your messaging system to fail. For more information about disk mirroring and disk striping with parity, see "Achieving Fault Tolerance by Using RAID" in the Windows Server 2003 Deployment Kit.
To implement a RAID configuration, it is recommended that you use only a hardware RAID product rather than software fault tolerant dynamic disk features.
The following sections discuss four primary implementations of RAID: RAID-0, RAID-1, RAID-0+1, and RAID-5. Although there are many other RAID implementations, these four types serve as an adequate representation of the overall scope of RAID solutions.
RAID-0 is a striped disk array. Each disk is logically partitioned in such a way that a "stripe" runs across all the disks in the array to create a single logical partition. For example, if a file is saved to a RAID-0 array, and the application that is saving the file saves it to drive D, the RAID-0 array distributes the file across logical drive D. In this example, the file spans all six disks.
RAID-0 disk array
From a performance perspective, RAID-0 is the most efficient RAID technology because it can write to all six disks simultaneously. When all disks store the application data, the most efficient use of the disks occurs.
The drawback to RAID-0 is its lack of fault tolerance. If the Exchange mailbox databases are stored across a RAID-0 array and a single disk fails, you must restore the mailbox databases to a functional disk array and restore the transaction log files. In addition, if you store the transaction log files on this array and you lose a disk, you can perform only a "point-in-time" restoration of the mailbox databases from the last backup.
RAID-0 is not a recommended solution for Exchange.
RAID-1 is a mirrored disk array in which two disks are mirrored.
RAID-1 disk array
RAID-1 is the most reliable of the three RAID arrays because all data is mirrored as it is written. You can use only half of the storage space on the disks. Although this may seem inefficient, RAID-1 is the preferred choice for data that requires the highest possible reliability.
If your goal is to achieve high reliability and maximum performance for your data, consider using RAID-0+1. RAID-0+1 provides high performance by using the striping benefits of RAID-0, while ensuring redundancy by using the disk mirroring benefits of RAID-1.
RAID-0+1 disk array
In a RAID-0+1 disk array, data is mirrored to both sets of disks (RAID-1) and then striped across the drives (RAID-0). Each physical disk is duplicated in the array. If you have a six-disk RAID-0+1 disk array, three disks are available for data storage.
RAID-5 is a striped disk array, similar to RAID-0 in that data is distributed across the array. However, RAID-5 also includes parity. There is a mechanism that maintains the integrity of the data stored on the array so that if one disk in the array fails, the data can be reconstructed from the remaining disks. Therefore, RAID-5 is a reliable storage solution.
RAID-5 disk array
However, to maintain parity among the disks, 1/n disk space is forfeited (where n equals the number of drives in the array). For example, if you have six 9-GB disks, you have 45 GB of usable storage space. To maintain parity, one write of data is translated into two writes and two reads in the RAID-5 array. Thus, overall performance is degraded.
The advantage of a RAID-5 solution is that it is reliable and uses disk space more efficiently than RAID-1 and RAID-0+1.
When planning your RAID solution, you must consider performance, reliability, disk capacity, and cost. Although cost and capacity are important, the primary considerations are the performance and reliability of your storage system.
The following table compares the cost and reliability of the four different RAID solutions.
Comparing the cost and reliability of RAID solutions
|RAID solution||Number of drives||Cost||Reliability|
10 9-GB disks
2 45-GB disks
20 9-GB disks
11 9-GB disks
You evaluate cost by calculating the number of disks needed to support your array. A RAID-0+1 implementation is the most expensive because you must have twice as much disk space as you need. However, this configuration yields much higher performance than the same-capacity RAID-5 configuration, as judged by the maximum read and write rates. RAID-1 is the least expensive because it requires only two 45-GB drives to store 90 GB of data. However, using two large disks results in much lower throughput.
- Reliability and performance
You assess reliability by evaluating the impact that a disk failure would have on the integrity of the data. RAID-0 does not implement any kind of redundancy, so a single disk failure on a RAID-0 array requires a full restoration of data. RAID-0+1 is the most reliable solution of the four because two or more disks must fail before data is potentially lost.
You assess performance by fully testing the various RAID levels in a test environment. You must select your hardware, RAID levels, and storage configuration to meet or exceed the performance levels required by your organization. To test the performance of your Exchange storage subsystem, use Jetstress and other Exchange capacity tools. For information about the best practices for achieving the required levels of performance, reliability, and recoverability, see Best Practices for Configuring Exchange Back-End Storage.
It is recommended that you use a SAN for the storage of your Exchange files. This configuration optimizes server performance and reliability.
|It is generally recommended that you use direct access storage device (DASD) or SAN-attached storage solutions, because this configuration optimizes performance and reliability for Exchange. Microsoft does not support network-attached storage solutions unless they meet specific Windows Logo requirements. For information about supported network-attached storage solutions, see "Network-Attached Storage Solutions" later in this topic.|
A SAN provides storage and storage management capabilities for company data. To provide fast and reliable connectivity between storage and applications, SANs use Fibre Channel switching technology.
A SAN has three major component areas:
Fibre Channel switching technology
Storage arrays on which data is stored and protected
Storage and SAN management software
Hardware vendors sell complete SAN packages that include the necessary hardware, software, and support. SAN software manages network and data flow redundancy by providing multiple paths to stored data. Because SAN technology is relatively new and continues to evolve rapidly, you can plan and deploy a complete SAN solution to accommodate future growth and emerging SAN technologies. Ultimately, SAN technology facilitates connectivity between multi-vendor systems with different operating systems to storage products from multiple vendors.
SAN storage solution
Currently, SAN solutions are best for companies and for Information Technology (IT) departments that need to store large amounts of data.
Although deployment cost can be a barrier, a SAN solution may be preferable because the long-term total cost of ownership (TCO) can be lower than the cost of maintaining many direct-attached storage arrays. Consider the following advantages of a SAN solution:
If you currently have multiple arrays managed by multiple administrators, centralized administration of all storage frees up administrators for other tasks.
In terms of availability, no other single solution has the potential to offer the comprehensive and flexible reliability that a vendor-supported SAN provides. Some companies can expect enormous revenue loss when messaging and collaboration services are unavailable. If your company has the potential to lose significant revenue as a result of an unavailable messaging service, it could be cost-effective to deploy a SAN solution.
Before you invest in a SAN, calculate the cost of your current storage solution in terms of hardware and administrative resources, and evaluate your company's need for dependable storage.
The following are advantages to implementing a SAN solution in your Exchange 2003 organization:
Exchange 2003 requires high I/O bandwidth that is supported only by a SAN-attached (sometimes referred to as channel-attached) storage array. In contrast, network storage solutions that rely on access to Exchange 2003 database files through the network stack can increase the risk of data corruption and performance loss.
Exchange 2003 also requires mailbox and public folders stores to exist on a drive that is local to the Exchange server drive. This requirement is met by SAN solutions that connect to Exchange servers through a local Fibre Channel connection. Other storage solutions that rely on a network redirector to process disk resources do not meet this requirement.
SANs are highly scalable, which is an important consideration for Exchange. As mail data grows and mailbox limits are continually challenged, you must increase storage capacity and I/O rates. As your organization expands, a SAN allows you to easily add disks.
It is recommended that you select a SAN solution that incorporates storage virtualization. Storage virtualization allows you to easily add disks and quickly reallocate that additional capacity to your Exchange servers. With storage virtualization, you can purchase additional storage disks in accordance with your capacity requirements and budget.
The scalable nature of SANs also makes it possible to expand your Exchange organization by adding servers. With SANs, you can connect multiple Exchange servers to multiple storage arrays and then divide the storage among them.
Through the use of mirroring and shadow copy backups that use Volume Shadow Copy service, backup, recovery, and availability are all enhanced with a SAN.
For information about the how you can enhance your storage solution using Volume Shadow Copy service, see Best Practices for Using Volume Shadow Copy Service with Exchange Server 2003.
Through replication strategies that include replicating data from a SAN in one site to a SAN in a different site, you can protect the data in your organization in the event of a site failure. Moreover, if you implement geographically dispersed clustering in coordination with your site replication strategy, your messaging system will be operational as well.
For information about replicating data to alternative sites, see "Using Multiple Physical Sites" in System-Level Fault Tolerant Measures.
For information about geographically dispersed clustering, see "Geographically Dispersed Clustering" in Planning Considerations for Clustering.
Network-attached storage refers to products that use a server-attached approach to data storage. In this approach, the storage hardware connects directly to the Ethernet network through Small Computer System Interface (SCSI), Fibre Channel connections, or the recently added support for Internet SCSI (iSCSI). A network-attached storage product is a specialized server that contains a file system and scalable storage. In this model, data storage is decentralized. The network-attached storage appliance connects locally to department servers, and therefore, the data is accessible only by local servers.
For information about iSCSI, see the Microsoft Storage Technologies - iSCSI Web site.
For information about support for iSCSI in Exchange, see Microsoft Knowledge Base article 839686, "Support for iSCSI technology components in Exchange Server."
|Exchange 2003 has local data access and I/O bandwidth requirements that network-attached storage products do not generally meet. Improper use of Exchange 2003 software with a network-attached storage product may result in data loss, including total database loss.|
For more information about network-attached storage solutions specific to Exchange 2003, see Microsoft Knowledge Base article 839687, "Microsoft support policy on the use of network-attached storage devices with Exchange Server 2003."
For information about network-attached storage solutions for Exchange 5.5 and later versions, see Microsoft Knowledge Base article 317173, "Exchange Server and network-attached storage."
For information about comparing SAN and network-attached storage solutions, see Microsoft Knowledge Base article 328879, "Using Exchange Server with Storage Attached Network and network-attached storage devices."
If your Exchange organization must function continuously, even in the event of a site disaster, you can increase the reliability of your Exchange data by implementing data replication technologies that replicate application data to one or more remote sites. Such data replication technologies allow you to synchronize SANs in multiple sites over long distance fiber or IP networks. The replication involved in these solutions is either synchronous or asynchronous.
For information about how you can use multiple physical sites to add redundancy to protect your application data, see "Using Multiple Physical Sites" in System-Level Fault Tolerant Measures.
For information about how you can use remote site replication in conjunction with Windows Clustering, see "Geographically Dispersed Clustering" in Planning Considerations for Clustering.
Solutions that include synchronous replication technologies can help you achieve 100 percent data reliability. Synchronous replication technologies write to both storage platforms (both the primary and the replicated storage device) before reporting to the operating system that the write was successful. Depending on the distance between the two storage platforms, this latency can be significant (+50 milliseconds). This increased latency creates a load on the server that severely affects the Exchange client experience. Specifically, the high write latency can cause excessive remote procedure call (RPC) queues and can potentially exhaust all 100 RPC threads (threads waiting on write I/O), thereby causing severe usability problems.
To minimize the negative effect on usability, you can reduce the number of users on both the server and the SAN. For example, consider an Exchange deployment that includes synchronous replication, where the user load is limited to only 1,000 users, even though both the server and SAN could support 4,000. Although synchronous replication has high data reliability, it does employ a significant I/O write performance penalty. This I/O write penalty is a critical factor in the number of users supported on a given platform.
|Solutions that use synchronous replication may be best served by using multiple two-processor servers, as opposed to implementing a consolidated model with servers that have four or eight processors. Server consolidation has proven to be reduced with synchronous replication technologies.|
Asynchronous replication does not have a negative effect on Exchange client performance because the replication writes are handled after the primary storage write is complete. The problem with asynchronous data replication is that it can take up to a minute (different for each SAN vendor) for the replication write to complete, thereby increasing the chances of data loss during a disaster. Asynchronous replication has no write performance penalty but is less reliable in terms of data reliability.
|If you select an asynchronous method, ensure that your disaster recovery procedures are well tested. Also, understand that there is a possibility for some data loss during a disaster. For this reason, asynchronous replication solutions are not recommended with geographically dispersed clusters.|
If you select a synchronous method, as with the implementation of any storage subsystem, deploy your data replication solution in a test environment to validate the design. The documentation included with the Jetstress tool includes information about how to test data replication solutions. For information about Jetstress, see Using Jetstress to Test Disk Performance.
You can download Jetstress at the Microsoft Exchange Server Jetstress Tool Web site.