Windows Server 2003 R2
Get Control of File System Resources with Windows Server 2003 R2
John Savill
At a Glance:
- Using the File Server Resource Manager
- Enforcing disk quotas in Windows Server
- Performing file screening
Windows Server 2003
Resource Management
Storage Reports
In December 2005, we saw the release of Windows Server 2003 R2. With this release comes a large number of new and improved features, including enhanced print management
simpler hardware and Storage Area Network (SAN) management, federated services for Web-based application access between organizations, enhanced UNIX integration, new tools for application developers, and new file system capabilities. (And since the core has remained untouched, Windows Server™ 2003 R2 requires minimal retesting.)
The new file system capabilities are significant, and deserve a close look. In fact, I think they will be one of the primary reasons for R2 adoption among existing Windows Server 2003 installations.
In general, the new file system capabilities fall into two categories. The first is the File Server Resource Manager (FSRM), which delivers an amazing advancement in the Windows Server file management capability. It gives administrators a great deal more power to offer a more reliable and organized file service while cutting down on the associated management time.
The second area is the enhanced Distributed File System Replication, or DFS Replication, engine. This allows information to be stored consistently over a geographically diverse environment using minimal bandwidth.
For the purposes of this article, I will focus on the FSRM. For more information about the improved Distributed File System Replication engine, see Alan von Weltin's article in this issue.
The Cost of Data
Storage is getting cheaper and disk sizes are going up. With this larger, cheaper storage infrastructure comes the mindset of "just buy more disk space." But what some people fail to realize is that physical disk space is generally not the most expensive component of data storage. In any company, data is one of the most important assets and, therefore, must be securely stored and easily recoverable. The management of data can often cost much more than the storage devices the data is stored on. So while the disks may be cheap, data maintenance is not. And as more data is created and retained, the cost of managing it increases accordingly.
This increase in total cost of ownership (TCO) led to a number of features that users wanted to see in the next release of Windows Server:
- Better reporting capabilities for identifying how storage is being used
- The ability to define quotas on folders and volumes that consider actual disk space usage
- More granular controls for specifying how storage can be used
Three core components make up the File Server Resource Manager, each addressing one of these points and each in many ways building upon the last.
Reporting
How much space is being wasted on duplicate files? How is the disk space used by file type? What are the least and most used files? Before Windows Server 2003 R2, there was no way to quickly gather this information and present it in a readable way. Mechanisms to ascertain information about disk space usage were minimal at best. The new storage report system, however, addresses this by defining a number of key scenario reports that can be customized and run on an as-needed basis or on a defined schedule. The built-in storage reports are defined via the XSL format (at this time it is not recommended that you modify them). The included storage reports are listed in Figure 1.
Figure 1 Windows Server 2003 R2 Storage Reports
| Report | Purpose |
| Duplicate Files | Finds files that share a common size and last modified date. Once identified, this information can be used to reclaim wasted disk space. |
| File Screening Audit | Scans the screening audit events to find violations of screening policies. |
| Files by File Group | Finds disk usage by type of file. Can be configured to report on all files or only files of certain types, such as audio and video or e-mail. |
| Files by Owner | Usage by file owner for all or selected users. Can be configured to report on only certain file name patterns, such as *.mp3. |
| Large Files | Lists all files above a certain definable size and optionally based on a certain file name pattern. |
| Least Recently Accessed Files | Lists all files that have not been accessed for a definable number of days, optionally matching a certain file name pattern. |
| Most Recently Accessed Files | Lists all files that have been accessed within a definable number of days, optionally matching a certain file name pattern. |
| Quota Usage | Lists all quotas and percent currently used. |
The storage reports can be output in a variety of formats including HTML, Dynamic HTML (which allows re-sorting of data within the Web browser), CSV, text, and XML. Once created, reportsare saved (at C:\StorageReports by default) and can optionally be delivered to a mailbox or distribution list.
Figure 2 shows how you create a scheduled report task. According to this configuration, a "Files by File Group" report will be output in DHTML, XML, and text formats. The Schedule tab is where you indicate when to run the report and the Delivery tab lets you specify an e-mail address to send the report to. Note that when viewing a Dynamic HTML report, you may see an ActiveX® warning at the top of Internet Explorer, since some scripting is used to power the dynamic content.
Figure 2** Creating a Scheduled Report Task **
Figure 3 shows a small part of a generated report; you can see the selected folder's usage by file type. If you want more detailed reports or want to perform more analysis on a folder or volume, you can simply output reports to XML and then import them into an analytical program of your choice for more complex calculations and reporting.
Figure 3** Generated Report **
Cluster installations are fully supported in R2. This means that if a scheduled report is defined on one node in the cluster, it will be accessible from the other nodes that may share disks, since the entire FSRM configuration is stored on the actual volume (more on this later).
Reports are often scheduled to run after hours (to avoid taxing systems during the business day) and then copied to a Web site or SharePoint® site where they can be viewed by, for example, departmental heads who may be charged by IT for their disk space usage. Now these people can easily view a detailed, sortable report of how disk space is being used. On-demand report generation should be used with care; if run during peak business hours, report generation can degrade the performance of the file server.
Quotas
Windows 2000 Server introduced disk quotas, which allow you to define a maximum amount of logical disk space on a per-volume and per-user basis. If a hard limit was exceeded, the user was unable to write additional information; if a soft limit was exceeded, an event log entry was written.
This quota system had limitations. Each user had his own entry and for shared areas the quota did not work. In addition, the quota was for an entire disk. On file servers, users may have access to many different shares and areas on a single volume; therefore, different quota limits should be possible at that level of granularity. For shared areas, a folder limit should be shared between anyone who has write permissions.
The new quota management system of FSRM provides this level of granularity. The system is based on physical disk usage rather than logical usage, and a number of notifications can be defined to fire in the event of a quota breach or when a certain percentage of the quota is reached. As with the quota system on Windows 2000 Server, this feature is only available on NTFS-formatted non-removable volumes.
As I mentioned, the Windows 2000 Server quota implementation is based on logical disk usage while the new FSRM is based on physical usage. But what do the terms "logical" and "physical" actually mean? The logical size is not the actual space used on the physical disk but is instead the size of the data itself. If I compress a 10MB (logical size) Word document, it may only use 350KB (physical size) on disk. On Windows 2000 Server, this compressed file would use 10MB of the user's quota; under the FSRM quota solution, only 350KB of the quota is used. This should encourage users to be more diligent about compressing information, as it will directly affect the amount of data they can store.
Another big change is that quotas are defined on a per-volume or per-folder basis—no user or group is specified. To restrict who can write to a folder, NTFS permissions should be used. Quotas only control how much can be written, NTFS controls who can write. Once a quota is applied to a folder or volume, anyone with write permission is restricted to that total limit.
FSRM quotas are highly flexible. A quota can be configured as hard or soft. A hard quota strictly enforces the quota limit. If an I/O request will exceed the quota, the request will be canceled. A soft quota, on the other hand, is not enforced and will allow continued I/O beyond the quota size. Both quota types allow the following actions to be triggered when the quota is reached.
E-Mail Message A quota can be set to send a customized message to a defined administrator and/or the user performing the I/O action. The message can include the quota used in MB, the percentage used, and so on, as shown in Figure 4. The e-mail message can even outline your organization's quota policies. By contrast, if the user simply receives an Access Denied message, he may not understand what the actual problem is.
Figure 4** Configuring a Quote-Threshold E-Mail Message **
Event Log Quotas can also be configured to write a warning to the application event log. The warning can contain configurable content based on a number of variables.
Command A powerful option is to run a script or command with a defined set of arguments when a quota is exceeded. The script or command can be run in the Local Service, Network Service, or Local System context.
Report You can also configure the FSRM to call a storage report and e-mail it to the user when a quota is surpassed. This would then give the user an opportunity to free up some space. You should use caution with this option, being aware of the resources required to run any reports that may be called. My preference is to run reports at night, outside of core hours, and use the e-mail message action to refer users to a network location to view usage reports. In many scenarios, however, the overhead of sending a report should be minimal.
These options provide you with a lot of flexibility. You can, for instance, specify a set of actions that warn users when they've reached 85 percent of their quota, and another set of more definitive actions when they've hit 100 percent, This approach can help you avoid a lot of reconfiguration on the various folders and volumes on which quotas are required.
The available use of templates can also help you to define actions that are to be performed at various percentage states of a quota. These templates can be quickly applied to volumes and folders.
Microsoft has provided a number of customizable templates (see Figure 5). Most everything that can be done through the GUI can also be configured using the command-line tools, and some very sophisticated combinations are possible. Once the templates are defined, a new quota entry is easily created by selecting Create Quota from the context menu of the Quotas node of Quota Management. The Create Quota dialog box (see Figure 6) allows the selection of a template or a custom set of criteria. Templates can be exported/imported using the command-line tools, which will aid in sharing templates between multiple machines.
Figure 5** Configuring a Quota Template **
At the top of Figure 6 notice that the quota can be created in one of two ways: Create quota on path, or Auto apply template and create quotas on existing and new subfolders. The common choice is to select Create quota on path, which applies the template or custom properties onto the selected quota path. The second option is useful on folders that may contain subfolders that need their own quotas. When Auto apply is selected, all existing subfolders have the quota applied individually; as new folders are created, they automatically have their own copy of the selected quota applied. This is shown in Figure 7.
Figure 6** Templates and Custom Properites **
Notice in Figure 7 that I created only the first quota, C:\Shared\Users\*; all the others were automatically applied as new folders were created. A very cool feature: if you modify a template, you can apply this change to all instances where the template is in use. For example, if I change the limit from 200MB to 500MB in the template and select the appropriate option to propagate this change to derived uses of the template, all the entries in Figure 7 would be changed to a 500MB limit.
Figure 7** Automatically Assigning Quotas **
File Screening
These quota capabilities are handy, but have a major failing. They assume that end users are using the allotted disk space as the administrator intended. But while the administrator views storage space as a place to save corporate data, many users see "their" space as a place to save their all-important MP3 collection. A quota will not prevent users from storing 500MB of ABBA tunes.
While quotas define how much data can be stored, file screens define what that data can be. As with quotas, file screening is performed in real time. Any attempt to write an illegal file type will dispatch an Access Denied error message to the client.
Nearly everything I discussed regarding quotas applies to file screening. File screen templates define what types of file categories (audio, video, images, and so on) should be blocked. The defined file groups contain all the popular file extensions for a group type. These file groups can be modified and new ones defined as needed. Any combination of the four actions—e-mail, event log, command execution, storage report—I discussed in terms of quotas can be configured to fire if a user attempts to store an illegal file type.
Also like quotas, file screening supports two modes: active and passive. Active mode will prevent an illegal file type from being written to the volume or folder, whereas passive mode will allow the file to be created while still running the defined set of actions (such as paging a very large security guard to escort the user off the company premises).
Exceptions can be defined to allow certain file groups to be stored. For example, at the root of a share, you may block all audio, video, and image files. You could then create an Images subfolder and create an exception to allow image files to be stored there. Users would be unable to write image files anywhere but in the Images subfolder (see Figure 8). This allows you to both define and enforce a very organized file storage environment.
Figure 8** File Screening **
If a file screen rule is applied to a folder that already contains illegal content, the files will not be deleted. They can still be read but not modified (even renamed) in any way. They can, of course, be deleted.
Finally, remember that file screening relies on file extensions. It is not designed to stop the determined user from renaming all of their MP3 files using different extensions in order to subvert the system.
Installation
So how do you get these features? The FSRM is not installed by default. You can install it on the server using the Add or Remove Programs Control Panel applet (see Figure 9) or by adding the File Server role using the Configure Your Server Wizard.
Figure 9** Adding the File Server Resource Manager **
Once installed, FSRM requires a restart. This is because the FSRM actually consists of four distinct services, two of which are loaded by the operating system as kernel-mode mini-filter file system drivers. These implement part of the file system before the operating system has completed startup. A reboot is necessary to ensure the drivers are correctly placed in the I/O stack in case legacy (non-mini-filter framework) drivers are present on the system.
Let's take a look at the four services that make up FSRM:
DataScrn Datascren.sys is a kernel-mode mini-filter file system driver that implements the file screening checks in real time on configured volumes and folders.
Quota Quota.sys is a kernel-mode mini-filter file system driver that implements the quota checks in real time on configured volumes and folders.
SrmSvc Srmsvc.dll is an application-type service that starts automatically on system startup and runs as a standalone process. It is visible as File Server Resource Manager.
SrmReports Srmhost.exe is an application-type service that runs as a standalone process. It can be started manually or when scheduled reports are configured to run. The service is visible as File Server Storage Reports Manager.
The File Server Resource Manager, a Microsoft Management Console (MMC) snap-in, is automatically installed to implement and manage the various facets of the FSRM subsystem. Configuration for FSRM is stored in the System Volume Information\SRM folder for each volume. This configuration is read and processed by the drivers at startup, along with information from the main SrmSvc service once it has started. The configuration is then cached in memory. If the quota or file screen entries change, the SrmSvc service notifies the mini-filter of the new entry set. Likewise, if the quota mini-filter file system driver detects that a folder has been created or deleted, SrmSvc updates any quota entries that may be affected.
Online Resources
- Managing Disk Quotas in Windows Server 2003 and Windows XP
- Windows Server 2003 R2 Roadmap
- Professor Windows—R2 Better than C3-PO
- Microsoft File Server Resource Manager IT Value Card
You never need to access this system volume information. All configuration is done using the MMC snap-in and the command-line tools. Configuration is backed up by NTBACKUP and other backup utilities that take advantage of the FSRM VSS writer.
Conclusion
The amount of storage space you manage will only continue to grow as your company gathers ever-increasing amounts of data. To cope with this, you need more efficient ways to manage your company's storage infrastructure. FSRM provides new tools that allow you to better understand how storage space is being used throughout your organization, and the capability to better enforce rules to control the use of disk space.
John Savill is Director of Technical Infrastructure for Geniant. He is a CISSP, a Security and Messaging MCSE on Windows Server 2003, a six-time MVP, and a Krav Maga instructor. He is also the author of Windows Server 2003 Active Directory Design and Implementation (Packt Publishing, 2005).
© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.