Note on IT

Published: December 10, 2004

The Microsoft IT group regularly uses the Diskpart.exe utility to dynamically increase disk volume capacity on servers running Microsoft® Windows Server� 2003 and Microsoft Windows® 2000 Server. The Diskpart.exe utility is an integral component of the Microsoft IT "just in time" storage model because it extends disk volumes, without downtime or application interruption, thereby increasing availability and decreasing costs.

Introduction

The Microsoft Information Technology (Microsoft IT) group regularly uses the Diskpart.exe utility to dynamically increase disk volume capacity on servers running the Microsoft Windows Server 2003 and Microsoft Windows 2000 Server operating systems.

Contained within Windows Server 2003 and Windows 2000 Server, the Diskpart.exe utility provides Microsoft IT with the means to implement a "just-in-time" storage model. The just-in-time storage model allows Microsoft IT to increase disk volume capacity when requested by internal customers, without downtime or application interruption. By making Diskpart.exe an integral component of their Storage Utility service, Microsoft IT can proactively manage storage usage and performance. In fact, Microsoft IT has reduced overall data center storage usage by up to 40 percent in its pilot project using Diskpart.exe and has reduced related costs.

This Note on IT describes the server-clone process formerly used by Microsoft IT (prior to using Diskpart.exe), third-party disk volume expansion tools, and Diskpart.exe usage information. It also describes how Microsoft IT uses the Storage Utility service to centrally manage and provision storage to internal Microsoft business units by using Diskpart.exe in an enterprise environment.

This Note on IT can assist customers planning to use Diskpart.exe to extend partitions dynamically on servers running Windows Server 2003 and Windows 2000 Server. Customers may identify similar scenarios in their own enterprise environments. In addition, this Note on IT may assist customers in determining how to implement a storage solution that will provide tangible benefits.

This Note on IT assumes readers are IT server administrators, disk storage architects, and technical implementers who are already familiar with Windows Server 2003 and Windows 2000 Server disk storage features and capabilities.

For more information about using the Diskpart.exe utility, see How to use Diskpart.exe to extend a data volume.

Note   For security reasons, the sample names of internal resources and other proprietary information used in this document do not represent actual names used within Microsoft. In addition, the contents of this document describe how Microsoft IT runs its enterprise data center. The procedures and processes included in this document are not intended to be prescriptive guidance on how to run a generic data center.

Former Server-Clone (Copy) Process

Prior to the advent of disk volume and partition expansion tools, Microsoft IT employed one basic strategy for managing user requirements to increase disk capacity, called the "server-clone" (or copy) process. When a user's storage requirements were greater than the available disk storage capacity of the server, the server-clone process provided IT disk storage architects and technicians with the following options for increasing the disk storage capacity:

• Purchase a new server. With adequate storage provided by the new server, the IT technician configured the storage and then copied the data across the network at an appropriately scheduled time.
• Rebuild the server. The IT technician installed new disk drives on an available server to build a larger-capacity server and then configured the storage and copied the data at a scheduled time. Each server rebuild required approximately 30 to 100 hours to complete.

Drawbacks

The drawbacks of using the server-clone process to increase disk storage capacity included:

• Severe customer impact. Backup and rebuild and restore activities often required approximately 4 to 50 hours of application downtime.
• Higher costs. The purchasing of new disk drives, new servers, and storage systems such as tape back-up systems increased costs and resource requirements.
• Over-provisioning of storage. Because repeating the process was complex and costly, disk storage architects using the server-clone process provisioned the disk storage based on the data owner's anticipated high-growth rate over a three year server lifecycle. As a result, disk storage utilization decreased from a high of 70 percent to an average of 40 percent.
• Over-committed resources. Due to the over-provisioning of storage, additional data center resources were required to support excess servers, network infrastructure, and staff. These additional requirements led to 60 percent of data center resources being over committed—often to maintain systems that were not even in use.

Third-Party Disk Volume Expansion Tools

Several years ago hardware storage-array technology made the expansion of disk volume capacity possible. With this technology, IT organizations were able to increase disk storage by using third-party tools that supported volume expansion.

Drawbacks

The drawbacks of using third-party tools to increase disk volume capacity included:

• The downtime when applications and servers are offline or are unavailable during the partition-growth operation.
• The expense associated with licensing fees for proprietary software normally costs from $400 to $1000 US for each server. This expense is required in addition to the typical cost of server software.

Diskpart.exe for Partition Expansion

Using Diskpart.exe, Microsoft IT is able to increase disk volume capacity as needed without application interruption, without an extensive server-clone process, and without additional licensing fees. Any non-boot partition can be expanded into available disk space with Diskpart.exe.

Important   Verify that your hardware array controllers support upgrading the disk drives and growing the hardware-based array without application interruption.

Benefits

Using Diskpart.exe to extend partitions dynamically offers these benefits:

• Time and resource savings. The 24 hours of downtime normally associated with the server-rebuild process for an upgrade can be reduced to an approximately three-hour, hands-on process to upgrade the server and increase the volume size—giving you more storage with less effort and fewer resources.
• Ongoing support savings. In addition, subsequent volume and partition growth becomes a one-hour hands-on process to complete—a time-saving process when compared to the approximately 30 to 100 hours required by the server-clone process.

Alternatives for Increasing the Partition Size

There are two ways to increase the size of a partition: concatenation and restriping (Diskpart.exe extends partitions increased by restriping).

Concatenation of Data

Concatenation links data partitions together to form larger partitions as shown in the following figure. This method of partition growth offers simple, low-cost data growth. However, concatenation isolates performance within each partition and does not offer fault tolerance.

Before there were tools such as Diskpart.exe that permitted extending a partition, partitions had to be concatenated.

Caution   If any of the concatenated partitions are lost, all data may also be lost.

If your browser does not support inline frames, click here to view on a separate page.

Figure 1   Data partitioning using concatenation

Restriping of Data

A striped volume contains data that is interleaved across two or more physical disks. The data on this type of volume is allocated alternately and evenly to each of the physical disks. When data is partitioned by using restriping, the RAID disk controller increases volume size by accessing the physical disk drives as contiguous space as shown in the following figure.

When data is restriped, data access is spread over multiple physical hard disks, increasing both space and performance. Restriping reduces operational effort and is more reliable than concatenation. Diskpart.exe can be used to extend a partition into the restriped disk space.

Important   Check with the storage vendor to ensure that your hardware array controllers support restriping and your storage system supports extending hardware-based RAID volumes dynamically.

If your browser does not support inline frames, click here to view on a separate page.

Figure 2   Data partitioning using restriping

Extending volumes in this way offers significant performance benefits over alternative methods of disk expansion.

Important   Diskpart.exe only supports NTFS basic primary partitions (logical volumes). Logical partitions with extended volumes are not allowed. Dynamic partitions are not supported. Diskpart.exe cannot support a volume that has been converted from a FAT file system to an NTFS file system.

Diskpart.exe Usage

Contained in Windows Server 2003 and Windows 2000 Server, the Diskpart.exe utility gives Microsoft IT a way to implement a just-in-time storage model that can dynamically increase disk volume capacity as requested by the data owners. Most third part tools do not provide this functionality, or instead offer disk volume expansion tools as an additional service.

Caution   Windows Server 2003 and Windows 2000 Server each use a different version of Diskpart.exe. To learn more about the two versions of Diskpart.exe, see How to use Diskpart.exe to extend a data volume.

Diskpart.exe is a text-mode command interpreter that manages objects (disks, partitions, or volumes). You can use Diskpart.exe to create a storage configuration by using a remote session, through direct input at the command prompt, or by using scripting. The following samples show how to use Diskpart.exe to extend disk volumes.

Diskpart.exe by Command Prompt

In this example the IT administrator accesses Diskpart.exe at the command prompt by typing DISKPART at the root directory. Then the IT administrator uses Diskpart.exe to check the volume size and disk storage that is currently available and to extend the volume size.

(In the following example, user input commands appear entirely capitalized, for example, "LIST DISK").

Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

C:\> DISKPART

Microsoft DiskPart version 5.2.3790
Copyright (C) 1999-2001 Microsoft Corporation.
On computer: MCSS2

DISKPART> LIST DISK

  Disk ###  Status      Size     Free     Dyn  Gpt
  --------  ----------  -------  -------  ---  ---
  Disk 0    Online         8 GB      0 B
  Disk 1    Online      5114 MB      0 B
  Disk 2    Online       210 GB    10 GB
  Disk 3    Online        20 GB      0 B
  Disk 4    Online        20 GB      0 B

DISKPART> SELECT DISK 2

Disk 2 is now the selected disk.

DISKPART> LIST PART

  Partition ###  Type              Size     Offset
  -------------  ----------------  -------  -------
  Partition 1    Primary            200 GB    32 KB

DISKPART> SELECT PART 1

Partition 1 is now the selected partition.

DISKPART> EXTEND

DiskPart successfully extended the volume.

Diskpart.exe and the Disk Management Snap-In

The Diskpart.exe utility contains an additional set of actions that are supported by the Disk Management snap-in. However, in contrast to other Disk Management snap-in functions, which prohibit you from inadvertently performing actions that may result in data loss, Diskpart.exe actions permit explicit control of partitions and volumes. For this reason it is recommended that you use these Diskpart.exe actions cautiously.

Caution   Once you use Diskpart.exe to increase the disk volume allocation, you cannot reverse the process and decrease the volume size. Diskpart.exe and most RAID arrays that can extend storage do not provide you with the option of undoing the process. For this reason, planning is crucial to ensure that disk space is not over-provisioned.

Diskpart.exe Scripting

Diskpart.exe supports scripted operations. Scripting is recommended when a procedure for increasing disk storage on servers is used more than once. For more information on the use of scripting with Diskpart.exe, see Diskpart, Microsoft Windows Server 2003.

Storage Utility Service

Today, a number of solutions are available to provide disk storage for server applications. Microsoft IT uses a combination of the following solutions:

• Direct Attached Storage (DAS). DAS consists of SCSI-based RAID controllers purchased from hardware vendors (single-server usage).
• Storage Area Network (SAN). A SAN is a private shared storage network (island) that is grouped for a single application, for example, a Marketing database.
• Storage Utility. The Storage Utility service is used to centrally manage and provision storage to all internal Microsoft business units (data owners).

Of these disk storage solutions, Microsoft IT uses the Storage Utility service to implement the just-in-time storage model. The Storage Utility service has two essential requirements. First, this service must have the capability to increase storage whenever the data owner requests it through the use of a SAN. Second, the Storage Utility service is implemented by using Diskpart.exe to dynamically increase partition size, thereby allowing disk storage growth on the fly—with no application downtime and no extra costs as shown in the following figure.

Figure 3   Volume expansion flow chart

Monitoring Disk Usage

For the Storage Utility server to work properly, it is important to determine the current volume usage and the degree of volume expansion the data owner anticipates. Careful planning with the data owner is essential to avoid creating a volume that is larger than necessary. If a volume is over-provisioned the only true recovery method is to recreate the volume.

The Storage Utility server uses custom queries in Microsoft Systems Management Server (SMS) 2003 to check partition usage. SMS provides alerts when the disk storage for a data owner reaches over 90 percent usage.

SMS is implemented by using an SMS agent on the Storage Utility server. The SMS agent monitors the storage usage for each server through the SMS console. The storage usage information obtained by SMS is issued on a custom report as shown in the following figure.

Figure 4   SMS storage usage report

When servers exceed the 90 percent target usage, Microsoft IT discusses the overage with the affected data owners on a case-by-case basis, giving them the opportunity to increase their storage allocation according to application requirements. By using Diskpart.exe, Windows Server 2003 and Windows 2000 Server can extend volumes to an upper-partition limit of two terabytes.

Data owners are given the following options for volume expansion:

• Auto-grow. Disk space is allocated automatically on a proactive basis. The data owner is not contacted until the upper-partition limit of two terabytes is reached.
• Growth with permission. The data owner is contacted when disk space usage exceeds 90 percent. The application owner confirms when additional disk space is needed.
• Never grow. The storage is never changed. When disk volume allocation occurs, it is reactive and on an after the fact basis, without customer consultation.

In closing, by using Diskpart.exe as an integral part of their Storage Utility service, Microsoft IT can proactively manage storage usage and performance. In fact, Microsoft IT has reduced overall data center storage usage by up to 40 percent in its pilot project using Diskpart.exe and has reduced related costs

For More Information

For more information about Windows Server 2003 storage solutions, go to http://www.microsoft.com/windowsserversystem/storage/default.mspx.

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/services/microsoftservices/howmsdoesIT.mspx

http://www.microsoft.com/technet/itsolutions/msit/default.mspx

For any questions, comments, or suggestions on this document, or to obtain additional information about Microsoft IT Showcase, please send e-mail to: showcase@microsoft.com

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Microsoft grants you the right to reproduce this White Paper, in whole or in part, specifically and solely for the purpose of personal education.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, email address, logo, person, place or event is intended or should be inferred.

© 2004 Microsoft Corporation. All rights reserved.

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESSED OR IMPLIED, IN THIS SUMMARY. Microsoft, Windows, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.