Planning the Staging Directory

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

The staging directory acts as a buffer by retaining copies of updated files until replication partners successfully receive them and move them into the target directory. The following sections describe the benefits and design considerations for staging directories. For an Excel spreadsheet to assist you in documenting the staging directory settings and location, see "FRS Configuration Worksheet" (Sdcfsv_2.xls) on the Windows Server 2003 Deployment Kit companion CD (or see "FRS Configuration Worksheet" on the Web at https://www.microsoft.com/reskit).

Why FRS Uses a Staging Directory

The staging directory acts as a queue for changes to be replicated to downstream partners. After the changes are made to a file and the file is closed, the file content is compressed, written to the staging directory, and replicated according to schedule. Any further use of that file does not prevent FRS from replicating the staging file to other members. In addition, if the file is replicated to multiple downstream partners or to members with slow data links, using a staging file ensures that the underlying file in the replica tree can still be accessed.

Note

• A new or updated file remains in the staging directory of a replica member with downstream partners even after the file has replicated to all members. The file is retained for seven days in order to optimize future synchronizations with new downstream partners. After seven days, FRS deletes the file. Do not attempt to delete any files in the staging directory yourself.

Staging Directory Size

The size of the staging directory governs the maximum amount of disk space that FRS can use to hold those staging files and the maximum file size that FRS can replicate. The default size of the staging directory is approximately 660 MB, the minimum size is 10 MB, and the maximum size is 2 TB. The largest file that FRS can replicate is determined by the staging directory size on both the upstream partner (the replica member that sends out the changed file) and downstream partners (the replica member that receives the changed file) and whether the replicated file, to the extent that it can be compressed, can be accommodated by the current staging directory size. Therefore, the largest file that FRS can replicate is 2 TB, assuming that the staging directory size is set to the maximum on upstream and downstream partners.

Managing Staging Directory Capacity

When FRS tries to allocate space for a staging file and is not successful, either because there is not enough physical disk space or because the size of the files in the staging directory has reached 90 percent of the staging directory size, FRS starts to remove files from the staging directory. Staged files are removed (in the order of the longest time since the last access) until the size of the staging directory has dropped below 60 percent of the staging directory limit. Additionally, staging files for downstream partners that have been inaccessible for more than seven days are deleted. As a result, FRS does not stop replicating if the staging directory runs out of free space. This means that if a downstream partner goes offline for an extended period of time, the offline member does not cause the upstream partner’s staging directory to fill with accumulated staging files.

Note

• In Windows Server 2003, FRS detects and suppresses excessive replication that could be caused when you use Group Policy to overwrite file permissions or when you use antivirus software that overwrites security descriptors. In these examples, writes to files cause no net content changes. When FRS detects a significant increase in the number of changes made to a file, FRS logs an event with ID 13567 in the File Replication service event log. Despite the FRS response to excessive replication caused by applications, services, or security policies, you should investigate the source of the excessive replication and eliminate it.

The fact that FRS removes files from the staging directory does not mean that the underlying file is deleted or will not be replicated. The change order in the outbound log still exists, and the file will eventually be sent to downstream partners when they process the change order. However, before replication can take place, the upstream partner must recreate the file in the staging directory, which can affect performance. Recreating the staging file can also cause a replication delay if the file on the upstream partner is in use, preventing FRS from creating the staging file.

A staging directory can reach 90 percent of its capacity in the following situations:

• More data is being replicated than the staging directory can hold. If you plan to replicate more than 660 MB of data, or if you expect that the largest file to replicate will be 660 MB or larger, you must increase the size of the staging directory. Use Table 2.7 for sizing guidelines.

• A slow outbound partner. Construct replica connections that have comparable bandwidth for all outbound partners. It is also a good idea to balance that bandwidth for inbound and outbound connections. If you cannot balance the bandwidth, increase the size of the staging directory.

• Replication is disabled. If a file is changed or added to the replica set while replication is disabled, the staging directory must be large enough to hold data awaiting replication during the off-time or else older staged files are removed. Replication still occurs, but less optimally.

Table 2.7 describes a range of sizing guidelines for staging directories. Use the numbers in Table 1.7 as a baseline for testing and then adjust the numbers as necessary for your environment. Also, note that the numbers in Table 1.7 apply to each replica set on a server. If a server contains multiple replica sets, you must follow these guidelines for each staging directory.

Table 2.7   Staging Directory Size Guidelines per Replica Set

Note

• The recommendations in Table 2.7 offer sufficient staging directory space for a worst-case scenario. In fact, the staging directory needs to be only as large as the largest 128 files, multiplied by the number of downstream partners that are concurrently performing a full synchronization against the replica member.

For more information about configuring the staging directory, see article Q329491, "Configuring Correct Staging Area Space for Replica Sets." To find this article, see the Microsoft Knowledge Base link on the Web Resources page at https://www.microsoft.com/windows/reskits/webresources.

For information about changing the size of the staging directory, see "Deploying FRS" later in this chapter.

Staging Directory Compression Considerations

FRS replica members running Windows 2000 Server with Service Pack 3 (SP3) or Windows Server 2003 compress the files replicated among them. Compression reduces the size of files in the staging directory on the upstream partners, over the network between compression-enabled partners, and in the staging directory of downstream partners prior to files being moved into their final location.

To maintain backward compatibility, servers running Windows 2000 Server with SP3 or Windows Server 2003 generate uncompressed staging files when they send changed files to downstream partners running Windows 2000 Server with Service Pack 2 (SP2) or earlier (or to servers running Windows 2000 Server with SP3 or Windows Server 2003 that have compression explicitly disabled in the registry).

To accommodate environments that contain a combination of compression-enabled and compression-disabled partners, the server originating or forwarding changes must generate two sets of staging files for each modified file or folder: one compressed version for servers running Windows 2000 Server with SP3 or Windows Server 2003, and one uncompressed version for partners that have compression explicitly disabled or that are running Windows 2000 Server with SP2 or earlier. To mitigate the generation of two sets of staging files, you can take one of the following steps:

• Confirm that you have sufficient disk space and an appropriately sized staging directory on all replica members.

• Explicitly disable compression in the registry until all members of the replica set are running Windows 2000 Server with SP3 or Windows Server 2003. For information about disabling replication, see article Q288160, "Error Message: Error Compressing Data," in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at https://www.microsoft.com/windows/reskits/webresources.

• Rapidly deploy Windows 2000 Server with SP3 or Windows Server 2003 on all members of the replica set, especially if you are replicating a large amount of content.

Choosing the Staging Directory Location

You can choose the location of the staging directory when you use the Distributed File System snap-in to configure a replica set. By default, all replica sets on the server use the same staging directory, and the staging directory space limit applies to all staging directories on the local server. When the staging directory is shared among multiple replica sets, any one replica set can cause the staging directory to reach 90% capacity, at which point FRS begins to delete the oldest staging files for any of the replica sets.

To simplify staging directory management and any future recovery procedures, it is recommended that you configure replica sets in one of the following ways:

• Give each replica set its own staging directory, and place the staging directories on the same volume. This solution provides good recoverability while minimizing cost.

• Give each replica set its own staging directory, and put each staging directory on its own volume. This solution provides better recoverability, but at a higher cost.

Do not store the staging directory on a volume where NTFS disk quotas are enabled.

Improving FRS Performance

To distribute disk traffic, store the staging directory, FRS logs, the FRS jet database (Ntfrs.jdb), and replica trees on separate disks. Using separate disks for the log files is especially important when you configure FRS logging by using a high severity level. For the best replication performance, distribute disk I/O by placing the FRS logs and the staging directory on separate disks from that of the operating system and the disks containing the replica tree. In a hub and spoke topology, using separate disks is especially recommended for the hub servers.

Important

• If you store the FRS jet database on a disk that has its write cache enabled, FRS might not recover if power to the drive is interrupted and critical updates are lost. Windows Server 2003 warns you about this by adding an entry with Event 13512 in the File Replication service event log. To leave write caching enabled for improved performance and to ensure the integrity of the FRS jet database, use an uninterrupted power supply (UPS) or a disk controller and cache with a battery backup. Otherwise, disable write caching until you can install a backup power supply.

When moving the FRS logs and FRS jet database, be sure that NTFS disk quotas are disabled on the target volume. For more information about moving the FRS logs and FRS jet database, see article Q221093, "HOW TO: Relocate the NTFRS Jet Database and Log Files," in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at https://www.microsoft.com/windows/reskits/webresources.