SharePoint Online and OneDrive Migration User Guide

SharePoint 2013

We are in the process of combining the SharePoint Server 2013 and SharePoint Server 2016 content into a single content set. We appreciate your patience while we reorganize things. See the Applies To tag at the top of each article to find out which version of SharePoint an article applies to.

 

Applies to: OneDrive for Business

Topic Last Modified: 2015-12-11

This is a step by step guide on how to use the SharePoint Online Migration PowerShell cmdlets to migrate content from an on-premises fileshare or an on-premises SharePoint Server site to Office O365.

SharePoint Online Migration PowerShell cmdlets are designed to move on-premises content from file shares and SharePoint Server libraries and One Drive to SharePoint Online and OneDrive for Business. Requiring minimal CSOM calls, it leverages Azure temporary BLOB storage to scale to the demand of large migration of data content.

  • Supported Operating Systems: Windows 7 Service Pack 1,Windows 8,Windows Server 2008 R2 SP1,Windows Server 2008 Service Pack 2, Windows Server 2012, Windows Server 2012 R2,

  • PowerShell 4.0

NoteNote:
Permissions: You must be a site collection administrator on the site you are targeting.

  1. Uninstall all previous versions of the SharePoint Online Management Shell.

  2. Install from here: SharePoint Online Management Shell.

  3. Open SharePoint Online Management Shell and select Run as Administrator.

Create two empty folders before you start the migration process. These folders do not require a lot of disk space as they will contain only XML.

  1. Create a Temporary package folder

  2. Create a Final package folder

  1. Create an Azure storage account.

  2. Make note of your Azure Storage account name and key as you need them for subsequent steps.

# Queue is only container specified as new containers are automatically generated

# All container URIs get populated into $al variable output from Set-SPOMigrationPackageAzureSource cmdlet

$creds = (Get-Credential admin@contoso.com)

$sourceFiles = '\\fileshare\users\charles'

$sourcePackage = 'C:\migration\CharlesDocumentsPackage_source'

$targetPackage = 'C:\migration\CharlesDocumentsPackage_target'

$targetWeb = 'https://contoso-my.sharepoint.com/personal/charles_contoso_com'

$targetDocLib = 'Documents'

$azureAccountName = 'contosomigration'

$azureAccountKey = 'Insert Azure Account Key string Here'

$azureQueueName = 'migrationqueue'

After you determine your locations and credentials, the next step is to create a new migration package from a file share. There are two options in creating a content package: either from a fileshare or from a SharePoint Server site. Choose one of the following methods.

To create a content package from a fileshare, the New-SPOMigrationPackage command reads the list of content targeted by the source path and will generate XML to perform migration.

There are two required parameters to enter (others are optional) :

  • SourcefilesPath: points to the content you want to migrate

  • OutputPackagePath: points to your Temporary folder

Example:

# Create new package from file share

New-SPOMigrationPackage -SourceFilesPath $sourceFiles -OutputPackagePath $sourcePackage

To create a content package from an on-premises SharePoint Server site, the first step is to use the Export-SPweb cmdlet. This cmdlet exports a site, list or library from SharePoint Server.

We require the following parameters:

  • Identity: Specifies the URL or GUID of the Web to be exported.

  • Path: Specifies the name of the export file. Because we require that the NoFileCompression parameter is used, a directory must be specified.

  • NoFileCompression: Either enables or disables file compression in the export package. File compression must be disabled.

  • ItemUrl: Specifies the URL of the Web application, GUID, or object to be exported.

Example:

# Create new package from an on-premises SharePoint Server site

Export-SPWeb [-Identity] <SPWebPipeBind> -Path <String> [-ItemUrl <String>] [-NoFileCompression <SwitchParameter>]

After you have created the content package, the ConvertTo-SPOMigrationTargetedPackage command converts the xml generated in your temporary folder. It saves a new set of targeted migration package metadata files to the target directory. This is the final package.

NoteNote:
Your target site collection administrator credentials are used to gather data to connect to the data site collection.

There are six required parameters to enter (others are optional)

  • TargetwebURL: points to the destination Web

  • SourceFilesPath: points to the you want to migrate

  • SourcePackagePath: points to your Temporary package folder

  • OutputPackagePath: points to your final package folder

  • TargetDocumentLibraryPath: the path to your destination library

  • Credentials: SPO credential that has admin rights to the destination site

Example:

# Convert package to a targeted one by looking up data in target site collection

ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFiles -SourcePackagePath $sourcePackage -OutputPackagePath $targetPackage -TargetWebUrl $targetWeb -TargetDocumentLibraryPath $targetDocLib -Credentials $creds

In this step, you create the migration package containers in Azure storage using your storage account credentials using the Set-SPOMigrationPackageAzureSource command. The package files are uploaded to the container, a snapshot of the files completed, and then it returns the connection strings to a PowerShell variable.

There are five required parameters to enter (others are optional)

SourceFilePath: points to the content you want to migrate

SourcePackagePath: points to your final package folder

AccountName: the name of the Azure storage account

Accountkey: the key to the Azure storage account

AzureQueueName: The name of the Azure queue BLOB (optional but recommended)

TipTip:
Take the output of the following command and store it in a variable for the next command.
$temporaryVariable = Set-SPOMigrationPackageAzureSource

Example:

# Create azure containers and upload package into them, finally snapshotting all files

$al = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFiles -SourcePackagePath $targetPackage -AzureQueueName $azureQueueName -AccountName $azureAccountName -AccountKey $azureAccountKey

$al|fl

The final step is to use the Submit-SPOMigrationJob command to create a new migration job in the target site collection, and returns a GUID representing the JobID.

There are three required parameters to enter (others are optional):

  • TargetwebURL: point to the Web of the destination

  • MigrationPackageAzureLocations: the temporary variable from the previous command

  • Credentials: SPO credential that has Site Collection Administrator rights to the destination site

Example:

# Submit package data to site collection to create new migration job

Submit-SPOMigrationJob -TargetWebUrl $targetWeb -MigrationPackageAzureLocations $al -Credentials $creds

# Example file share packaging, conversion, upload and job submission script
# Queue is only container specified as new containers are automatically generated
# All container URIs get populated into $al variable output from Set-SPOMigrationPackageAzureSource cmdlet
$creds = (Get-Credential admin@contoso.com)
$sourceFiles = '\\fileshare\users\charles'
$sourcePackage = 'C:\migration\CharlesDocumentsPackage_source'
$targetPackage = 'C:\migration\CharlesDocumentsPackage_target'
$targetWeb = 'https://contoso-my.sharepoint.com/personal/charles_contoso_com'
$targetDocLib = 'Documents'
$azureAccountName = 'contosomigration'
$azureAccountKey = 'Insert Azure Account Key string Here'
$azureQueueName = 'migrationqueue'
# Create new package from file share without security (faster)
New-SPOMigrationPackage -SourceFilesPath $sourceFiles -OutputPackagePath $sourcePackage
# Convert package to a targeted one by looking up data in target site collection
ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFiles -SourcePackagePath $sourcePackage -OutputPackagePath $targetPackage -TargetWebUrl $targetWeb -TargetDocumentLibraryPath $targetDocLib -Credentials $creds
# Create azure containers and upload package into them, finally snapshotting all files
$al = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFiles -SourcePackagePath $targetPackage -AzureQueueName $azureQueueName -AccountName $azureAccountName -AccountKey $azureAccountKey
# This displays the return Azure location
$al|Format-List
# Submit package data to site collection to create new migration job
Submit-SPOMigrationJob -TargetWebUrl $targetWeb -MigrationPackageAzureLocations $al -Credentials $creds

After the job is submitted, only Azure and SPO are interacting to fetch and migrate the content into the destination. This process is Timer Job Based, which means it’s a queue on a first come first served basis. This does not prevent other jobs from being queued up by the same person. There is a potential of a 1 minute delay if there are no other jobs running.

You can check the status of your job by viewing the real time updates posted in the Azure storage account queue.

SPO Migration job status in the Azure storage account queue

You can look into the manifest container in the Azure Storage for logs of everything that happened. At this stage, it is now safe to delete those containers if you don’t want to keep them as backup in Azure. If there were errors or warnings, .err and .wrn files will be created in the manifest container.

Viewing SPO Migration logs in the manifest container in Azure storage

$userName = "admin@contoso.onmicrosoft.com"
$adminSite = "https://contoso-admin.sharepoint.com"
 $sourceFilePath = "D:\data\documents\"
$packagePath = "d:\data\documentPackage"
$targetWebUrl = "https://contoso.sharepoint.com/sites/finance"
 $spoPackagePath = "d:\data\documentPackageForSPO"
$targetLibrary = "Shared Documents"
 $azureAccountName = "acctName"
$azureAccountKey = "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000=="
$azureQueueName = "contosoMigrationQueue"
 $cred = Get-Credential $userName
New-SPOMigrationPackage -SourceFilesPath $sourceFilePath -OutputPackagePath $packagePath
 ConvertTo-SPOMigrationTargetedPackage -TargetWebUrl $targetWebUrl -SourceFilesPath $sourceFilePath -SourcePackagePath $packagePath -OutputPackagePath $spoPackagePath -TargetDocumentLibraryPath $targetLibrary -Credentials $cred
$azureSource = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFilePath -SourcePackagePath $spoPackagePath -AccountName $azureAccountName -AccountKey $azureAccountKey -AzureQueueName $azureQueueName
Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azureSource -Credentials $cred 

 

Description

Recommendation

Package size

2-4 GB

File size

2 GB

Target size

Target site should remain non-accessible to users until migration is complete

SharePoint Online limits

SharePoint Online and OneDrive for Business: software boundaries and limitsSharePoint Online: software boundaries and limits

 

Resource

Default/Limit

TB per storage account

500 TB

Max size of single blob container, table, or queue

500 TB

Max number of blob containers, blobs, file shares, tables, queues, entities, or messages per storage account

Only limit is the 500 TB storage account capacity

Target throughput for single blob

Up to 60 MB per second, or up to 500 requests per second

Show: