Understanding the Impact of Large Sector Media for IT Pros

Updated: June 20, 2012

Applies To: Windows Server 2008 R2, Windows Server 2008 R2 with SP1

In recent years, the sector size of storage media has typically been 512 bytes. However, the storage industry has recently started transitioning to media with larger sector sizes. With these new types of media, there are key impacts to performance, resiliency, functionality, and even to deployment scenarios. This document is targeted at IT Pros and system administrators, and it discusses how these new types of media may affect your customers, technologies, deployment considerations, and how you can be prepared for this transition.

Areal densities are increasing yearly, and with the recent arrival of 3 TB disks from various manufacturers, the error correction mechanisms that are used to handle decreasing Signal-to-Noise-Ratio (SNR) are becoming space inefficient—that is, there is an increased amount of overhead required to ensure the media is usable. One of the storage industry’s solutions to improve this error correction mechanism is to introduce a different physical media format in the form of a larger physical sector size on the media.

In this article, we have discussed how large sector media impact many common deployment scenarios. This included how Read-Modify-Write impacts performance and resiliency, and how these disks can introduce some difficult scenarios, and why they can potentially cause issues with software and end-users. The storage industry is rapidly transitioning to media with larger sector sizes, and very soon customers will not be able to purchase disks with traditional 512-byte sector sizes.

This document is meant as an aid to IT Pros and system administrators to help ease this transition. You should start a conversation with your customers, fellow IT Pros, and your software and hardware vendors to talk about the large sector transition and what the impact is to the scenarios which are important to you. Please feel free to contact Microsoft Customer Service and Support if you are experiencing any issues in your deployments with 512e-type disks on the following operating system versions:

  • Windows 7 with Microsoft KB 982018

  • Windows 7 SP1

  • Windows Server 2008 R2 with Microsoft KB 982018

  • Windows Server 2008 R2 SP1

  • Windows Vista with Microsoft KB 2553708

  • Windows Server 2008 with Microsoft KB 2553708

For information on Microsoft’s current support policy on Advanced Format Media, refer to Microsoft KB Article 2510009. For more information on Advanced Format drives, you should contact your storage vendor.

This article includes the following sections:

  • Logical vs. physical sector size

  • Types of large sector media

  • How emulation works: Read-Modify-Write

  • The Resiliency Impact of Read-Modify-Write

  • Overall Windows support for large sector media

  • Windows technology support for 512e media

  • Common deployment scenarios with large sector media

  • Additional references

Logical vs. physical sector size

As a result of the compatibility layer, there are two types of sector sizes: logical and physical.

Logical sector: The unit that is used for the logical block addressing (LBA) for the media. We can also think of it as the smallest unit of write that the storage can accept. This unit is the “block” in logical block addressing.

Physical sector: The unit of granularity for which read and write operations to the device are completed in a single operation. We can think of this as the unit of atomic write, and may also presume that this is the optimal write size for the media.

The Windows operating system provides standard APIs for applications to query the physical sector size.

This change in the media format presents possible incompatibility with the current software, hardware, and support available in the market today. As a temporary compatibility layer, the storage industry will initially introduce disks that emulate a regular 512 byte sector disk, but make available information about the true sector size through standard commands.

Types of large sector media

The storage industry is quickly ramping up efforts to transition to this new type of storage. This new storage technology is called “Advanced Format” for storage medium with a 4 KB physical sector size. There will be two types of media released to the market:

  1. 4 K native (512n): This media has no emulation layer, and it directly exposes 4 KB as its logical and physical sector size. This media is not currently supported with Windows and most other operating systems, although Microsoft is conducting an investigation into the feasibility of supporting this type of media in the future and will issue a Knowledge Base article when appropriate.

  2. 512-byte emulation (512e): This media has an emulation layer as discussed in the previous section, and it exposes 512 bytes as its logical sector size (similar to a regular disk today). However, 512e makes available information about its physical sector size, which is 4 KB. Although this media accepts writes in the granularity of the logical sector size, applications that issue those writes may experience a number of unexpected issues as a result of the Read-Modify-Write cycle discussed in the following section. This is what is currently being introduced to the market from several storage vendors.

The overall issue with this type of media is that the majority of application and operating systems do not understand the existence of the physical sector size. This can result in a number of issues, which are discussed in the following sections.

How emulation works: Read-Modify-Write

The physical storage media can only be written or rewritten in units of the physical sector size. For a 512e-type disk, the physical sector size is 4 KB, and the logical sector size is the traditional 512 bytes. Therefore, writes that are not performed at this unit will require additional steps. The following diagram illustrates the required steps for the storage device to complete a write to the logical sector.

Figure 1  Write process

The Resiliency Impact of Read-Modify-Write

One of the most important issues with 512e media is the resiliency impact of the Read-Modify-Write cycle with some implementations of Read-Modify-Write. On some disks, if the cycle was interrupted by a power loss, there is a chance that the data contained within the physical sector will be lost (or the eight logical sectors that are contained within the physical sector will be lost). The details for this include:

  • Most applications maintain a structured storage file to keep information associated with an individual record within a unit at least the size of a single sector. This is because most applications understand that units of a sector can fail at any time or the write may be interrupted, and they do not want to lose more than one record for each instance of failure.

  • Unless applications have been updated, the unit of loss that most applications use with 512e media is the logical sector (512 bytes). This can be problematic because the real unit of loss is the physical sector (4 KB). Therefore, when an application’s I/O pattern results in Read-Modify-Write within the storage device; it presents a window for failure.

  • An incident can occur where the act of another application causes a Read-Modify-Write cycle that can potentially cause your data to be lost—even if your application is not running. This can happen because your data and the other application’s data could be located within the same physical sector.

Windows technology support for 512e media

Note

Storage devices that are labeled as “Advanced Format” or “512 byte Emulation” devices, but do not report the physical sector size through standard ATA and SCSI commands, are not supported by the Windows operating systems.

The following table details support for 512-byte emulation media under certain scenarios that use Windows operating systems. This table outlines the support for some features of the Windows operating systems, and it does not imply any applications running on these versions of Windows will have support for large sector media. In addition, support relies on the storage driver and the storage device being able to report the sector size information.

Scenario Windows XP, Windows Server 2003, Windows Server 2003 R2 Windows Vista, Windows Server 2008 Windows 7, Windows Server 2008 R2

Physical sector size reporting

Not supported

Supports MSAHCI

Supports ATAPORT

Intel AHCI supported as of Microsoft KB 2553708.

Note
A driver update for Windows Vista and Windows 7 users is available separately from Intel.

Supports Storport as of Microsoft KB 2553708

Note

The Storport miniport driver needs to support SBC3 READ_CAPACITY(16), and the system administrator needs to set a registry key as stated later in this article.

Supports USBStor as of Microsoft KB 2553708

Supports MSAHCI

Supports ATAPORT

Intel AHCI supported as of SP1

Note

A driver update for Windows Vista and Windows 7 users is available separately from Intel.

Supports Storport as of SP1

Note

The Storport miniport driver needs to support SBC3 READ_CAPACITY(16), and the system administrator needs to set a registry key as stated later in this article.

Supports USBStor as of SP1

Partition alignment

Not supported

The first partition is aligned as of Windows Vista SP1 and Windows Server 2008 RTM

The first partition is aligned as of Windows Vista SP1 and Windows Server 2008 RTM

Optimal File System Performance and Reliability

Compromised

Non-optimal

File system performance and reliability enhanced as of SP1

Extensible Storage Engine

Not supported

Supported with Microsoft KB 2553708

Supported as of SP1

Reporting utility for end-users

Not available

Supported with Microsoft KB 2553708

Supported with Microsoft KB 982018

HyperV

Not available

For a support statement about using Hyper-V with large sector drives, see article 2515143 in the Microsoft Knowledge Base.

For a support statement about using Hyper-V with large sector drives, see article 2515143 in the Microsoft Knowledge Base.

Common deployment scenarios with large sector media

This section discusses some of the common deployment storage scenarios for IT Pros. These scenarios are intended as guidelines for the set of issues that you might encounter, and they include some recommendations for mitigating these issues.

Scenario 1: Imaging and rollout

One of the most common scenarios for an IT Pro is reimaging a computer. Frank is a system administrator for a medium-sized company, and he regularly needs to reload computer images as part of system upgrades or to replace bad hard disks. To save time, Frank makes a single computer image for a specific model and loads it on several computers. However, recent shipments of hard drives from the manufacturer or the OEM have switched to the new Advanced Format type of disk, and Frank is noticing some performance and functionality issues when he loads the image and when the computer is running.

Note

We recommend that you initialize and partition the system partition with Windows PE 2.1 or later, and that your system image is using Windows 7 Service Pack 1 or Windows Server 2008 R2 Service Pack 1. (Minimally, you need to apply the hotfix that is described in article 982018 in the Microsoft Knowledge Base into RTM media). We also strongly recommend performing the initial system imaging on an Advanced Format device.

One of the main causes of issues related to 512e devices is that the application, the operating system, or the file system is not able to retrieve the physical sector size of the storage device to ensure that any I/O that is targeting the device is aligned. The Advanced Format device needs to report the physical sector size. Therefore, Microsoft does not support Advanced Format devices. Following are recommendations for potential issues.

Potential Issue Sample Causes Sample Resolutions

Performance

Primary partition unaligned

  • Make sure to use Window PE 2.1 or later to initialize and partition the system partition.

  • If you are using a non-Microsoft partition utility, contact your vendor for support.

 

File system writes are not optimal for 4 K media

  • Ensure that you upload the hotfix described in Microsoft Knowledge Base article 982018 or Service Pack 1 for Windows 7 and Windows Server 2008 R2 into your deployment image.

  • If you are using a Storport-based storage controller, ensure that you have set the registry key described in Microsoft Knowledge Base article 982018.

  • If all else fails, try re-creating the image on a system where the storage controller is set to AHCI in the BIOS.

 

User application not optimized for 4 K media

Ensure that you upload the hotfix described in Microsoft Knowledge Base article or Service Pack 1 for Windows 7 and Windows Server 2008 R2 into your deployment image to introduce the proper infrastructure for physical sector size reporting.

  • If you are using a Storport-based storage controller, ensure that you have the latest driver from the vendor. Also, ensure that you have set the registry key as described in Microsoft KB article 982018.

  • If all else fails, contact your software vendor for support. The software may need to independently provide support for this type of media.

Windows Update fails

The reported physical sector size of the system volume has changed

  • For Windows 7, ensure that you upload the hotfix described in Microsoft Knowledge Base article 982018 or Service Pack 1 for Windows 7 and Windows Server 2008 R2 into your deployment image. These upgrades include an updated version of Esent.sys, which addresses the Windows Update issue.

  • If you are using Windows Vista, you should upload the hotfix described in Microsoft Knowledge Base article 2553708 into your deployment image.

Scenario 2: Disks behind RAID

RAID is a common storage scenario for many deployments that is used for resiliency and performance, or to enable larger volumes of storage. The problem is that many RAID implementations do not report the physical sector size of the underlying storage device. This can result in performance and reliability issues similar to the issues that are experienced when the Windows operating system and applications running on Windows are not able to retrieve the physical sector size of the storage device. There are some RAID implementations that mitigate some of the performance and resiliency issues of 512e, and it is recommended that customers use these implementations. Some recommendations for addressing these issues are listed in the following table:

Issue Potential Mitigation

Performance

Utilize a RAID controller with a sufficient amount of cache RAM.

Applications failing to start after power loss

Utilize a RAID controller with a charged battery installed or a controller that features non-volatile cache (NVC). Alternatively, the user can install a uninterruptable power supply (UPS) on the system that is hosting the storage.

Unable to retrieve physical sector size

Contact your RAID controller manufacturer for support.

Scenario 3: Storage device running on top of Storport-based host bus adapters

Many servers utilize Storport-based host bus adapters (HBA). With Windows 7 SP1 and Windows Server 2008 R2 SP1, Microsoft introduced Storport support for physical sector size reporting. However, because this support was released as a hotfix, and to help reduce the risk of possible regressions in sending new commands, Microsoft requires that the system administrator set a registry key for each HBA that supports this new infrastructure.

To support physical sector querying with Storport-based HBAs, you need the following setup:

  • Windows 7 SP1, Windows Server 2008 R2 SP1, or the hotfix described in article 982018 in Microsoft Knowledge Base installed on Windows 7 RTM or Windows Server 2008 R2 RTM.

  • Windows Vista with Microsoft KB 2553708 or Windows Server 2008 R2 with Microsoft KB 2553708

  • An updated Storport Miniport driver from your HBA vendor that supports SBC-3 READ_CAPACITY(16). You should contact your HBA vendor for support information.

  • Registry key set as shown in the following image for each HBA:

    Figure 2  Registry key setting

Scenario 4: Changing physical sector sizes

Because there are varying levels of support for 512e media, there are many instances where the reported physical sector size of the underlying volume may change underneath the application. This can impact performance, resiliency, and functionality issues. You should consider the following scenarios where the reported physical sector size can change between application sessions:

  • The system hosting the application storage is migrated from a device with a 512-byte physical sector size to a storage device with a 4 KB physical sector size (or vice versa).

  • You copy your application storage to a RAID set from a directly attached disk (or vice versa). The reported physical sector size may change because RAID controllers might not report the physical sector size of the storage device. Therefore, the system might read “4 KB” in one session and “512 bytes” in another session (or vice versa).

  • The storage controller of the system is upgraded so that the current controller has support for physical sector size reporting, but the new controller does not (or vice versa). Because the storage controller changes, the Windows operating system needs the appropriate driver to support it. This issue commonly occurs when you are upgrading from the Microsoft Inbox ATA driver (MSAHCI) to a non-Microsoft Storport-based driver (or vice versa).

  • The mode of the storage controller of the system’s BIOS is changed. This issue can occur in all combinations of modes, such as AHCI, Legacy, IDE, Compatible, and RAID. Each mode requires a different storage driver. The Windows operating system needs the appropriate driver. Some drivers have support and others may not.

  • You are deploying system images to 512e media from an image that was created on a 512n media (or vice versa). The reported physical sector size of the image will be different than the reported physical sector size on the deployed system when it restarts.

This list is not comprehensive, but it describes some of the scenarios that can cause the reported physical sector size to change. Many applications may be unable to handle this change in the reported physical sector size. If you plan to migrate your application storage (as described in one of the previous scenarios), you should contact your application vendor to determine if that is a supported scenario.

If the application does not support the migration, and if the application refuses to start, you can attempt the following:

Warning

Make sure that you back up any data before attempting the following task because many applications behave differently and the steps can result in data loss.

Delete the application log files. You can delete the relevant log files and reinitialize the application. Then you can attempt to recover the application by using the Last Known Good Configuration option.

Scenario 5: The resiliency impact of Read-Modify-Write

Writing unaligned data to 512e media can result in the loss and/or corruption of data that is contained within the physical sector. This potential for data loss is generally only applicable to high-performance database-style applications. Some possible symptoms include:

  • Your application is unable to initialize its primary storage.

  • You experience data loss and/or data corruption.

  • You may notice errors in the application log files.

It is strongly recommended that you contact your application vendor to help determine the support statement for 512e media. Review the following list of suggestions to help prevent data loss or corruption.

Warning

Make sure that you back up your data before you attempt any of the following tasks because many applications behave differently and the steps can result in further data loss.

  • Delete the application log files. You can delete the relevant log files and reinitialize the application. Then you can attempt to recover the application by using the Last Known Good Configuration option.

  • Rebuild the primary application storage with a repair utility. Many applications have a repair utility that you can use to help the application recover from data loss. This utility helps the application continue to function properly.

  • Recover the primary application storage from a backup. If nothing else works, this should resolve your issue.

  • Utilize a storage device with power mitigation capability. You should contact your hard disk vendor to determine if your storage device has this capability.

If your high-performance application does not support 512e media with optimal reliability, we recommend that users install an uninterruptable power supply (UPS) to the system that is hosting the storage, or that they utilize storage devices that feature non-volatile cache (NVC), super capacitors, or other methods to help maintain the cache during power-outages.

Retrieving the logical and physical sector size for a volume

Windows includes a utility called fsutil that displays the sector size for a volume. Windows versions that support fsutil include:

  1. Windows Developer Preview

  2. Windows Server Developer Preview

  3. Windows 7 SP1 with Microsoft KB 982018

  4. Windows 7 with Microsoft KB 982018

  5. Windows Server 2008 R2 SP1 with Microsoft KB 982018 (v3)

  6. Windows Server 2008 R2 with Microsoft KB 982018 (v3)

  7. Windows Vista with Microsoft KB 2553708

  8. Windows Server 2008 with Microsoft KB 2553708

To retrieve sector size information, run fsutil from an elevated command prompt as follows:

fsutil fsinfo ntfsinfo <drive letter>

A 4K sector disk with 512-byte emulation has the Bytes Per Sector option set to 512, and the Bytes Per Physical Sector option set to 4096 as represented in the following screen.

A 4K native disk has both the Bytes Per Sector and Bytes Per Physical Sector options set to 4096 as represented in the following screen.

Note

If the Bytes Per Physical Sector option displays “Not Supported”, the storage driver may not support IOCTL_STORAGE_QUERY_PROPERTY, or there is an error in retrieving the information.

Additional references