DFS Replication: Copying Files to Preseed or Stage Initial Synchronization

 

Applies To: Windows Server 2008, Windows Server 2003 R2, Windows Server 2008 R2, Windows Server 2012 R2, Windows Server 2012

The topics in this guide describe how to preseed files for Distributed File System (DFS) Replication (also known as DFSR or DFS-R) to speed up initial synchronization when you set up DFS Replication, add a new replication partner, or replace a server. Preseeding is required when you set up the database cloning feature, which was introduced in Windows Server 2012 R2.

Distributed File System Replication synchronizes files between two or more servers. When you set up replication, the initial synchronization time drops significantly if DFS Replication can use files that are already in place on the destination volume.

DFS Replication was introduced in Windows Server 2003 R2. Methods described in this guide can be used to preseed files for DFS Replication from servers running Windows Server 2003 R2 and later operating systems to any server that supports DFS Replication, including servers running Windows Server 2012 R2 or Windows Server 2012. Not all methods support all of these operating systems; for guidance, see Comparison of tools, later in this topic. The methods can be used to preseed files from folders that are already replicated through DFS Replication or folders that will be replicated for the first time.

Note

This guide assumes that you have a basic knowledge of DFS Replication. For more information, see DFS Namespaces and DFS Replication Overview.

Overview of preseeding files for DFS Replication

DFS Replication is a state-based, multi-master file replication service that uses a database and the NTFS USN journal to track file changes. In its simplest configuration, two servers communicate with each other bidirectionally (writable) or unidirectionally (Read-only). For an individual file update operation, each server can operate in an upstream or downstream fashion.

When configuring replication on the second – and subsequent – servers, DFS Replication completes a two-phase process:

  • Phase 1: Initial build – DFS Replication scans all existing files and folders in the replicated folder and adds a record for each one to the database.

  • Phase 2: Initial sync – DFS Replication exchanges replicated file metadata - such as each file hash, unique identifier, and global version sequence number – with its partner server. Inbound, the downstream (destination) server replicates any missing files or files that have a different hash from the file on the upstream (source) server. If DFS Replication finds any files that have a matching hash on the same relative path on both servers, the file is not replicated downstream.

When you preseed file data on the downstream server, DFS Replication does not have to replicate those files inbound during initial synchronization. For larger data sets, this can save significant setup time and allow DFS Replication faster entry into the normal replication state, in which only new, changed, or modified files are replicated between servers.

Note

To speed up initial synchronization when you add a new replication partner or replace a server in an existing DFS Replication setup, you can preseed the replicated folder on the destination server and then export a clone of the DFS Replication database. If you both preseed the replicated folder and export a database clone, members of a replication group do not have to exchange files or metadata during replication setup. For more information about this feature, introduced in Windows Server 2012 R2, see Export a Clone of the DFS Replication Database.

For preseeding to work, however, you must copy files from the upstream servers to the downstream servers without changing their file hashes. A file hash created by DFS Replication incorporates the data stream, the alternate data stream, and the access control list (ACL) applied directly to that file. This ensures that any aspect of file changes replicates to all other nodes. Incorrectly preseeding files on a downstream server leads to slower replication than not preseeding, because DFS Replication takes the additional step of verifying every existing file, but then must discard and replicate each file.

Comparison of tools

The tool that is best suited for your environment depends on the environment, the operating systems on the source and destination servers, and the nature and size of the data sets that you want to preseed for DFS Replication. The following table compares three tools that can be used for preseeding: Robocopy, Windows Server Backup, and NTBackup. This guide provides step-by-step procedures for preseeding files with each of these tools.

Preseeding tool

Advantages

Limitations

Robocopy

  • Copies files directly from the source directory to the destination directory.

  • Can be used to preseed files from Windows Server 2003 R2 and later, through Windows Server 2012 R2.

  • Cannot copy files with exclusive locks. To effectively preseed, you must stabilize files on the source computer before copying begins.

  • Requires manual installation of the correct Robocopy version for the source or destination operating system.

Windows Server Backup

  • Easily installed feature of Windows Server 2012 R2, Windows Server 2012, Windows Server 2008 R2, and Windows Server 2008.

  • Copies files with exclusive locks.

  • Requires backing up to an interim storage location.

  • Cannot be used to preseed files from Windows Server 2003 R2. (Must be installed on both the source and destination server.)

  • On a source server running Windows Server 2008, Windows Server Backup can only back up volumes, not individual folders. However, individual folders can be restored.

Windows NTBackup

  • If you are accustomed to using NTBackup, provides a familiar tool for preseeding files from Windows Server 2003 R2 for DFS Replication in Windows Server 2008 R2 or Windows Server 2008.

  • Copies files with exclusive locks.

  • Requires backing up to an interim storage location.

  • Cannot be used to preseed files for DFS Replication in Windows Server 2012 R2 or Windows Server 2012.

Additional preseeding methods might be available for your storage devices and accompanying storage software:

  • Hardware options - Some SAN and NAS devices allow synchronous or snapshot-based data duplication mirroring onto another volume. For example, you might be able to copy a volume currently in use on the source server to a new volume attached to the destination server. Because the new volume is a block-level image of the previous volume, the data and files should be perfectly preseeded.

    Note

    Be aware that Storage Spaces in Windows Server 2012 R2 and Windows Server 2012 does not allow creation of new volumes using mirror disks that were evicted from a previous storage pool.

  • Third-party software options - Your backup or storage software vendors might offer additional options for preseeding files. Review documentation from your vendor for more information.

Important

If you plan to preseed data by using any non-Microsoft solution, you should test the solution and then validate the preseeded data on the destination server to ensure that the file hashes are identical on the source and destination servers. For more information, see Step 2: Validate Preseeded Files for DFS Replication.

In this guide

This guide provides detailed instructions for preseeding files for DFS Replication, validating the preseeded files, and troubleshooting any issues that occur during preseeding:

Additional resources

Content type

References

Product evaluation

What's New in DFS Replication and DFS Namespaces in Windows Server | DFS Namespaces and DFS Replication Overview

Operations

DFS Replication help and basic operational guidance | DFS Replication: Frequently Asked Questions (FAQ) | Robocopy [LH] (command-line reference) | Windows Server Backup Step-by-Step Guide for Windows Server 2008 R2 | Windows Server Backup Step-by-Step Guide for Windows Server 2008 | Export a Clone of the DFS Replication Database

Community resources

The File Services and Storage TechNet Forum | Storage Team at Microsoft - File Cabinet Blog | Ask the Directory Services Team blog | DFS Replication: Survival Guide (TechNet Wiki) | Windows Server Information Experience on Twitter