Export (0) Print
Expand All

Exchange Online migration performance and best practices

Exchange Online
 

Applies to: Exchange Online, Exchange Server, Exchange Server 2013

Topic Last Modified: 2015-06-04

There are many paths to migrate data from an on-premises email organization to Exchange Online in Office 365. When planning a migration to Exchange Online, a common question is about how to improve the performance of data migration and optimize migration velocity.

NoteNote:
The performance information listed in this topic doesn’t apply to Microsoft Office 365 service for dedicated subscription plans. For more information about Dedicated Plans, see Office 365 Dedicated Plans Service Descriptions.

Exchange Online is part of Office 365 and provides organizations with a cloud-based messaging solution based on Microsoft Exchange Server. Exchange Online supports several methods to migrate email, calendar, and contact data from your existing messaging environment to Office 365.

For more information on Office 365 networking and performance, head over to our performance and tuning section.

Frequently used migration methods

Migration Method Description Resources

IMAP Migration

You can use the Exchange Administration Center (EAC) or the Exchange Management Shell to migrate the contents of users' mailboxes from an IMAP messaging system to their Exchange Online mailboxes. This includes migrating your mailboxes from other hosted email services, such a Gmail or Yahoo Mail.

For more information, see IMAP migration.

Cutover Migration

Using a cutover migration, you migrate all on-premises mailboxes to Exchange Online over a few days. You would use this migration type if you plan to move your entire email organization to Office 365 and manage user accounts in Office 365. You can migrate a maximum of 2,000 mailboxes from your on-premises Exchange organization to Exchange Online using a cutover migration. The mail contacts and distribution groups in your on-premises Exchange organization are also migrated.

For more information, see Cutover Exchange migration.

Staged Migration

You use a staged migration if you plan to eventually migrate all your organization's mailboxes to Exchange Online. Using a staged migration, you migrate batches of on-premises mailboxes to Exchange Online over the course of a few weeks or months. Your goal would be to permanently move your email organization to Office 365.

For more information, see Staged Exchange migration.

Hybrid Deployment

A hybrid deployment offers organizations the ability to extend the feature-rich experience and administrative control they have with their existing on-premises Microsoft Exchange organization to the cloud. A hybrid deployment provides the seamless look and feel of a single Exchange organization between an on-premises Exchange Server 2013 or 2010 organization and Exchange Online in Microsoft Office 365. In addition, a hybrid deployment can serve as an intermediate step to moving completely to an Exchange Online organization.

For more information, see Exchange Server 2013 Hybrid Deployments`

3rd Party Migration

There are many tools available from third parties. They use distinctive protocols and approaches to conduct email migrations from email platforms like IBM Lotus Notes and Novell GroupWise.

Here are some third-party migration tools and partners that can assist with Exchange migrations from third-party platforms:

  • Binary Tree   Provider of cross-platform messaging migration and coexistence software, with products that provide for the analysis of and the coexistence and migration between on-premises and online enterprise messaging and collaboration environments based on IBM Lotus Notes and Domino and Microsoft Exchange and Microsoft SharePoint.

  • BitTitan   Provider of migration solutions to Exchange Online.

  • Dell   Provider of on-premises and hosted migration and coexistence software, including pre-migration analysis and complete user and application coexistence. Full-featured migrations from on-premises Microsoft Exchange, IBM Domino, Novell GroupWise, Zimbra and other environments to Microsoft Office 365, Exchange Online, and SharePoint Online.

  • Metalogix   Provider of migration solutions to Exchange Online and SharePoint Online.

  • SkyKick Provider of automated migration solutions to move on-premises Exchange, Gmail, POP3, IMAP, Lotus Notes to Microsoft Office 365. The end-to-end migration tools help partners with the sales, planning, migration, management and onsite phases of the migration project.

  • TransVault   Provider of migration solutions to Exchange Online.

The following table compares the observed performance results for the different migration methods for migrating mailboxes and mailbox data to Exchange Online in Office 365. These results based on internal testing and actual customer migrations to Office 365.

ImportantImportant:
Because of many differences in how and when these migrations were performed, your actual migration velocity might be slower or faster

 

Migration Method Office 365 user throttling Office 365 migration-service throttling Office 365 resource health-based throttling Observed average throughput per hour and per client (if applicable)

IMAP Migration

No

Yes

Yes

10-14 GB (20 concurrency)

Cutover Migration

No

Yes

Yes

10-14 GB (20 concurrency)

Staged Migration

No

Yes

Yes

10-14 GB (20 concurrency)

Hybrid Migration

No

Yes

Yes

10-14 GB per on-premises Exchange 2013 or 2010 CAS (MRS Proxy) with 20 concurrent moves1

Third party MAPI Migration

Yes

No

Yes

4-12 GB (20 concurrency) 3

Third party EWS Migration

No

Yes

Yes

5-10 GB (20 concurrency) 2

Client Uploading (From Outlook PST)

Yes

No

Yes

0.5 GB

1Observed single mailbox move throughput is in the 0.3–1.0 GB/hour range. Greater than 1000 MB/h per mailbox throughput rate can be achieved with a network that can sustain less than a 2% transient failure stall time and less than 100ms network latency. More concurrent mailbox migrations can be used to achieve higher data migration rates. Single mailbox move throughput will slow down when the on-premises CAS (MRSProxy) server is at hardware capacity, if the network bandwidth isn’t sufficient, or the network latency is too high. Consider adding more servers or temporarily improving network connectivity to increase migration velocity.

2Observed single EWS migration throughput is in the 0.2–0.5 GB/hour range. More concurrent migrations can be used to achieve higher data migration rates. For example, with 100 20 concurrent migrations, the overall throughput will be in the 20–5010-14 GB/hour range. Single EWS migration throughput will slow down when on-premises servers or the network is at capacity.

3Observed single MAPI migration throughput is in the 0.1-0.5 GB/hour range. More concurrent migrations can be used to achieve higher data migration rates. Single MAPI migration throughput will slow down when on-premises servers or the network is at capacity.

Email migration has several common factors that can affect migration performance.

The following table provides a list of common factors that affect migration performance. More details are covered in the sections on the individual migration methods.

 

Factor Description Example

Data source

The device or service that hosts the data to be migrated. Many limitations might apply to the data source because of hardware specifications, end-user workload, and back-end maintenance tasks.

Google Mail limits how much data can be extracted during a specific period of time.

Data type and density

Because of the unique nature of a customer’s business, the type and mix of mail items within mailboxes vary greatly.

One 4-GB mailbox with 400 items, each with 10 megabytes (MB) of attachments, will migrate faster than one 4-GB mailbox with 100,000 smaller items.

Migration server

Many migration solutions use a "jump box" type of migration server or workstation to complete the migration.

Customers often use a low-performance virtual machine to host the MRSProxy for hybrid deployments or a client PC non-hybrid migrations.

Migration engine

The data migration engine that is responsible for pulling data from the source server converts data if necessary, transmits the data over the network, and injects the data into the Exchange Online mailbox.

Microsoft Exchange Mailbox Replication Service (MRS) has its own capabilities and limitations.

On-premises Network Appliances

The end-to-end network performance—from the data source to Exchange Online client access servers—affects migration performance.

Firewall configuration and specifications on the on-premises organization.

Office 365 service

Office 365 has built-in support and features to manage the migration workload.

The user throttling policy has default settings and limits the overall maximum data transfer rate.

This section describes best practices for improving network performance during migration. The discussion is general because the biggest impact on network performance during migration is related to third-party hardware and Internet service providers (ISPs).

The Office 365 Network Analysis Tool is deployed to help analyze network-related issues prior to deploying Office 365 services:. Each of these instances is designed to test a particular region using test end-points in Office 365.

For more direct and thorough testing from client computers on your network to your Office 365 service, the Microsoft Exchange Client Performance Analyzer can be installed and run from multiple workstations and at regular intervals in silent mode.

Use the Microsoft Exchange Client Performance Analyzer to get a deeper understanding of your network connectivity with Office 365.

 

Factor Description Best practices

Network capacity

The amount of time it takes to migrate mailboxes to Exchange Online is determined by the available and maximum capacity of your network.

  • Identify your available network capacity and determine the maximum upload capacity.

  • Contact your ISP to confirm your allocated bandwidth and get details about restrictions, such as the total amount of data that can be transferred in a specific period of time.

  • Use tools to evaluate your actual network capacity. Make sure you test the end-to-end flow of data, from your on-premises data source to the Microsoft data center gateway servers.

  • Identify other loads on your network (for example, backup utilities and scheduled maintenance) that can affect your network capacity.

Network stability

A fast network doesn’t always result in fast migrations. If the network isn’t stable, data transfer takes longer because of error correction. Depending on the migration type, error correction can significantly affect migration performance.

Network hardware and driver issues often cause network stability problems. Work with your hardware vendors to understand your network devices and apply the vendor’s latest recommended drivers and software updates.

Network delays

Intrusion detection functionality configured on a network firewall often causes significant network delays and affects migration performance.

Migrating data to Exchange Online mailboxes relies on your Internet connection. Internet delays affect overall migration performance.

Also, users in the same company might have cloud mailboxes that reside in data centers in different geographical locations. Depending on the customer's ISP, there might be varying migration performance.

  • Evaluate network delays to all potential Microsoft data centers to help ensure that the result is consistent. (This also helps ensure a consistent experience for end users). Work with your ISP to address Internet-related issues.

  • Add IP addresses for Microsoft data center servers to your allow list, or bypass all migration-related traffic from you network firewall. For more information about the Office 365 IP ranges, see Office 365 URLs and IP address ranges.

For a deeper analysis of migrations within your environment, check out our move analysis blog post that includes a script to help you analyze move requests.

Office 365 uses various throttling mechanisms to help ensure security and service availability. The following three types of throttling can affect migration performance:

  • User throttling

  • Migration-service throttling

  • Resource health-based throttling

NoteNote:
The three types of Office 365 throttling don’t affect all migration methods.

User throttling affects most third-party migration tools and the client-uploading migration method. These migration methods use client access protocols, such as RPC over HTTP, to migrate mailbox data to Exchange Online mailboxes. These tools are usually used to migrate data from platforms such as IBM Lotus Domino and Novell GroupWise.

User throttling is the most restrictive throttling method in Office 365. Because user throttling is set up to work against an individual end user, any application-level usage will easily exceed the throttling policy and result in slower data migration.

Migration-service throttling affects all Office 365 migration tools. Migration-service throttling manages migration concurrency and service resource allocation for Office 365 migration solutions.

Migration-service throttling affects migrations performed by using the following migration methods:

  • IMAP migration

  • Cutover Exchange migration

  • Staged Exchange migration

  • Hybrid migrations (MRSProxy–based moves in a hybrid environment)

An example of migration-service throttling is controlling the number of mailboxes that are migrated simultaneously during simple Exchange migrations and IMAP migrations. The default value is 10. This means that a maximum of three mailboxes are migrated at any particular time during migration. You can increase the number of concurrent mailbox migrations for a migration batch in either the Exchange Control Panel or Windows PowerShell. To learn more about how to optimize this setting, see Manage migration batches in Exchange Online.

All migration methods are subject to the governance of availability throttling, but Office 365 service throttling doesn’t affect Office 365 migrations as much as the other types of throttling described in the previous sections.

Resource health-based throttling is the least aggressive throttling method and occurs only when there is a service availability issue that affects end users and critical service operations.

For example, if a service incident occurs during a hybrid migration, and the service degrades to the point where end-user performance is degraded, the hybrid migration will be queued until performance is recovered and the service returns to a level below the throttling threshold.

The following is an example from an Exchange migration statistics report that shows an entry caused when the service-throttling threshold is exceeded.

  • 1/25/2012 12:56:01 AM [BL2PRD0410CA012] Copy progress: 723/1456 messages, 225.8 MB (236,732,045 bytes)/416.5 MB (436,712,733 bytes).

  • 1/25/2012 12:57:53 AM [BL2PRD0410CA012] Move for mailbox '/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx' is stalled because DataMoveReplicationConstraint is not satisfied for the database 'NAMPRD04DG031-db081' (agent MailboxDatabaseReplication). Failure Reason: Database edbf0766-1f2a-4552-9115-bb3a53a8380b doesn’t satisfy constraint SecondDatacenter. There are no available healthy database copies. Will wait until 1/25/2012 1:27:53 AM.

  • 1/25/2012 12:58:24 AM [BL2PRD0410CA012] Request is no longer stalled and will continue.

Solution and practice

If you experience a similar situation, wait for the Office 365 service to recover. For more information, see the Service Health section in the Office 365 Portal.

This section describes factors that affect migrations using the IMAP, cutover or staged migration methods and best practices to improve migration performance.

The following table describes the impact on migration by the source servers in your current email organization and the best practices for mitigating the impact on migration.

 

Checklist Description Best practices

System performance

Data extraction is an intensive task. The source system needs to have sufficient resources, such as CPU time and memory, to provide optimal migration performance. During migration, the source system is often close to full capacity in terms of the regular end-user workload. If systems resources are inadequate, the additional workload that results from migration can affect end users.

Monitor system performance during a pilot migration test. If the system is busy, we recommend avoiding an aggressive migration schedule for the specific system because of potential migration slowness and service availability issues. If possible, enhance the source system performance by adding hardware resources and reducing the load on the system by moving tasks and users to other servers that aren’t involved in the migration.

For more information, see:

When migrating from an on-premises Exchange organization where there are multiple mailbox servers, we recommend that you create a migration-user list that is evenly distributed across multiple mailbox servers. Based on individual server performance, the list can be further fine-tuned to maximize throughput.

For example, if server A has 50 percent more resource availability than server B, it’s reasonable to have 50 percent more users from server A in the same migration batch. Similar practices can be applied to other source systems. Perform migrations when servers have maximum resource availability, such as after hours or on weekends and holidays.

Back-end tasks

Other back-end tasks that are running during migration time. Because it’s a best practice to perform migration after business hours, it’s common that migrations conflict with other maintenance tasks running on your on-premises servers, such as data backup.

Review other system tasks that might be running during migration. We recommend that you perform data migration when no other resource-intensive tasks are running.

Note   For customers using on-premises Microsoft Exchange, the common back-end tasks are backup solutions and Exchange store maintenance.

Throttling policy

It’s a common practice to protect email systems with a throttling policy, which sets a specific limit on how fast and how much data can be extracted from the system during a certain amount of time.

Verify what throttling policy is deployed for your email system. For example, Google Mail limits how much data can be extracted in a certain time period.

Depending on the version, Microsoft Exchange has policies that restrict IMAP access to the on-premises mail server (used by IMAP migrations) and RPC over HTTP access (used by cutover Exchange migrations and staged Exchange migrations).

To check the throttling settings in and Exchange 2013 organization, run the Get-ThrottlingPolicy cmdlet. For more information, see Exchange Workoad Management.

For more information about IMAP throttling, see Migrate Email from an IMAP Server to Exchange Online Mailboxes

For more information about RPC over HTTP throttling, see:

IMAP, cutover and staged migrations are cloud-initiated data-pull migration methods, so there’s no need for a dedicated migration server. However, the Internet-facing protocol hosts (IMAP or RPC over HTTP) function as the migration server for migrating mailboxes and mailbox data to Office 365. Therefore, the migration performance factors and best practices, described in the previous section about the data source server for your current email organization, also apply to the Internet edge servers. For Exchange 2013, 2010 and 2007 organizations, the client access server functions as a migration server.

For more information, see:

  1. Exchange 2013 Workload Management

  2. Exchange 2010: Client Access Server Counters

  3. Exchange 2007: Monitoring Client Access Servers

Solution and practice

For better migration performance, apply the same best practices as previously described.

IMAP, cutover, and staged Exchange migrations are performed by using the Migration dashboard in the Exchange Administration Center (EAC) Exchange Online. This tool is subject to Office 365 migration-service throttling.

Solution and practice

Customers can now specify migration concurrency (for example, the number of mailboxes to migrate simultaneously) by using Windows PowerShell. The default is 20. After you create a migration batch, you can use the following Windows PowerShell command to increase this to a maximum of 100.

Set-MigrationEndPoint <Identity> -MaxConcurrentMigrations <value between 1 and 100>

For more information, see Manage migration batches in Exchange Online.

NoteNote:
If your data source doesn’t have sufficient resources to handle all the connections, we recommend avoiding high concurrency. Start low as 10 and then increase this number while monitoring the data source performance to avoid end-user access issues.

Verification tests

Depending on the migration method, you can try the following verification tests:

  • IMAP migrations   Pre-populate a source mailbox with sample data. Then from the Internet (outside your on-premises network), connect to the source mailbox by using a standard IMAP email client such as Microsoft Outlook, and then measure network performance by determining how long it takes to download all the data from the source mailbox. The throughput should be similar to what customers can get by using the IMAP migration tool in Office 365, given that there are no other constraints.

  • Cutover and staged Exchange migrations   Pre-populate a source mailbox with sample data. Then from the Internet (outside of your on-premises network), connect to the source mailbox with Microsoft Outlook by using RPC over HTTP. Help ensure that you’re connecting by using cache mode. Measure network performance by checking how long it takes to synchronize all the data from the source mailbox. The throughput should be similar to what customers can get by using the simple Exchange migration tools in Office 365, given that there are no other constraints.

NoteNote:
There is some overhead during an actual IMAP, cutover, or staged Exchange migration, but the actual throughput should be similar to the results of these verification tests.

Office 365 resource health-based throttling affects migrations using the native Office 365 simple migration tools. See the Office 365 Resource Health-Based Throttling section.

Hybrid deployment migration supports the smooth migration between on-premises Exchange 2003, Exchange 2007, Exchange 2010, and Exchange 2013 servers and Exchange Online in Office 365.

Hybrid deployment migration is by far the fastest migration method to migrate mailbox data to Office 365. We have seen up to 100 GB/hour throughput during real customer deployments The follow table provides a list of factors that apply to native Office 365 hybrid migration scenarios.

 

Checklist Description Best practices

System performance

Data extraction is an intensive task. The source system must have sufficient resources, such as CPU time and memory, to provide better migration performance. At the time of migration, the source system is usually close to full capacity to serve regular end-user workload—additional migration workload sometimes even brings down end users' access because of a lack of system resources.

Monitor system performance during a pilot migration test. If the system is busy, we recommend avoiding an aggressive migration schedule for the specific system because of potential migration slowness and service availability issues. If possible, enhance the source system performance by adding hardware resources and reducing the load on the system by moving tasks and users to other servers that aren’t involved in the migration.

For more information see:

When migrating from an on-premises Exchange organization where there are multiple mailbox servers and multiple databases, we recommend that you create a migration user list that is evenly distributed across multiple mailbox servers and databases. Based on individual server performance, the list can be further fine-tuned to maximize throughput.

For example, if server A has 50 percent more resource availability than server B, it’s reasonable to have 50 percent more users from server A in the same migration batch. Similar practices can be applied to other source systems.

Perform migrations when servers have maximum resource availability, such as after hours or on weekends and holidays.

Back-end tasks

Other back-end tasks that are running during migration time. Because it’s a best practice to perform migration after business hours, it’s common that migrations conflict with other maintenance tasks running on your on-premises servers, such as data backup.

Review other system tasks that might be running during migration. We recommend that you perform data migration when no other resource-intensive tasks are running.

Note   For customers using on-premises Microsoft Exchange, the common back-end tasks are backup solutions and Exchange store maintenance.

Hybrid deployment migration is a cloud-initiated pull/push data migration, and an Exchange 2013 or Exchange 2010 SP3 hybrid server acts as the migration server. This is often overlooked, and customers use a low-scale virtual machine to act as the hybrid server. This results in poor migration performance

Best practice

In addition to the applying the best practices previously described, we’ve tested the following best practices which resulted in improved migration performance in real customer migrations:

  • Use powerful server-class physical machines instead of virtual machines for the Exchange 2013 and 2010 hybrid servers.

  • Use multiple hybrid servers that are behind the customer’s network load balancer.

For example, in real customer migrations, we have achieved consistent 30 GB/hour throughput by using the following configuration:

  • Network   500-MB outbound pipe to the Internet; internal network on 1 GB with a 10-GB fiber backbone.

  • Hardware   The specifications for the two client access/HUB (physical) servers are:

    • CPU: Intel® Xeon® CPU E5520 @ 2.27 GHz 2.26 GHz (two processors).

    • RAM: 24 GB.

    • Disks: Eight at 146 GB per disk. RAID 5 configuration = 960 GB total raw space.

  • MRSProxy   Configured with a concurrency of 100.

Hybrid deployment migration uses native Office 365 tools. It’s subject to Office 365 migration service throttling.

Exchange 2003 vs. Exchange 2007 SP2, Exchange 2010 SP3, and Exchange 2013 SP1

There is a key difference for the end-user experience when the migration is from Exchange 2003. Unlike Exchange 2007 SP2, Exchange 2010 SP3, and Exchange 2013 SP1, Exchange 2003 end users cannot access their mailboxes when their data is being migrated. Therefore, Exchange 2003 customers are usually more concerned about when to schedule migrations and the time required to migrate, especially when migration performance is low because of large mailbox sizes or a slow network.

Exchange 2003 migration is also very sensitive to interruptions. For example, in a real customer migration, during the migration of a 10-GB mailbox, a service incident occurred when the migration of the mailbox was 50 percent complete. The Office 365 client access server, which was processing the data migration, had to be restarted to resolve the issue. In this case, the migration of that mailbox had to be restarted, which meant that the customer had to migrate all 10 GB of data again. The migration couldn’t resume from the point when it stopped. However, Exchange 2007 SP2, Exchange 2010 SP3, and Exchange 2013 SP1 are able to resume migrations after interruptions.

Best practice

Some customers choose to do two-hop migrations for large and sensitive Exchange 2003 mailboxes:

  • First hop   Migrate mailboxes from Exchange 2003 to an Exchange 2010 SP3 server, which is usually the hybrid server. The first hop is an offline move, but it’s usually a very fast migration over a local network.

  • Second hop   Migrate mailboxes from Exchange 2010 SP3 to Office 365.

The second hop is an online move, which provides a better user experience and fault tolerance. This two-hop approach requires an Exchange 2010 license for the temporary on-premises Exchange 2010 user mailbox.

Mailbox Replication Service Proxy (MRSProxy)

MRS Proxy is the on-premises migration feature that works with the Mailbox Replication Service running on the Office 365 side. For more information, see Understanding Move Requests.

Best practice

It’s possible to configure the maximum number of MRSProxy connections for the on-premises Exchange 2010 SP3. Run the following Windows PowerShell command.

Set-WebServicesVirtualDirectory -Identity "EWS (Default Web Site)" -MRSMaxConnections <number between 0 and unlimited; default is 100>
NoteNote:
For most customer migrations, it’s unnecessary to change the default MRSMaxConnections value. If you need to protect the source server from being overwhelmed by the migration load, customers can reduce the number of connections. This setting is per MRSProxy server. If a customer has two MRSProxy servers, each set to 10 connections, they'll get 20 (2 x 10) as the total number of MRSProxy connections. For more information about configuring the MRSProxy service in your on-premises Exchange 2010 organization, see Start the MRSProxy Service on a Remote Client Access Server.

NoteNote:
For most customer migrations, it’s unnecessary to change the default MRSMaxConnections value. If you need to protect the source server from being overwhelmed by the migration load, customers can reduce the number of connections. This setting is per MRSProxy server. If a customer has two MRSProxy servers, each set to 10 connections, they'll get 20 (2 x 10) as the total number of MRSProxy connections. For more information about configuring the MRS Proxy service in your on-premises Exchange 2010 organization, see Start the MRSProxy Service on a Remote Client Access Server.

Verification tests

For Exchange 2013 and 2010 customers, testing the network performance for hybrid migrations can be done by performing multiple test mailbox migrations. Alternatively, you can migrate actual user mailboxes with the -SuspendWhenReadyToComplete option to get an indication of migration performance. When testing is complete, remove the move request to avoid affecting end users.

For more information about Exchange 2013 move requests, see New-MoveRequest.

For more information about Exchange 2010 move requests, see New-MoveRequest.

Office 365 resource health-based throttling affects migrations using the Office 365 hybrid deployment migrations. See the Office 365 resource health-based throttling section above for more details.

For general information about getting status information for move requests, see View Move Request Properties.

In the Office 365 service, there is a key difference from normal on-premises Exchange 2010 organization when performing move requests. In Office 365, the migration queue and the service resources allocated for migrations are shared among tenants. This affects how move requests are handled in each stage of the move process.

There are two types of move requests in Office 365:

  • Onboarding move requests   New customer migrations are considered onboarding move requests. These requests have regular priority.

  • Data center internal move requests   These are mailbox move requests initiated by data center operation teams. These requests have a lower priority because the end-user experience isn’t affected if there’s a delay in the move request.

  • Queued move requests   This status specifies that the move has been queued and is waiting to be picked up by the Microsoft Exchange Mailbox Replication service. For Exchange 2003 move requests, users can still access their mailboxes at this stage.

    Two factors influence which request will be picked up by Mailbox Replication service:

    • Priority   Queued move requests with a higher priority are picked up before lower-priority move requests. This helps ensure that customer-migration move requests always get processed before data center internal move requests.

    • Position in the queue   If move requests have the same priority, the earlier the request gets into the queue, the earlier it will be picked up by the Mailbox Replication service. Because there might be multiple customers performing mailbox migrations at the same time, it’s normal that new move requests remain in the queue before they’re processed.

      Often, the time that mailbox requests wait in the queue before being processed isn’t considered during migration planning. This results in customers not being allocated enough time to complete all planned migrations.

  • In-progress move requests   This status specifies that the move is still in progress. If this is an online mailbox move, the user will still be able to access the mailbox. For offline mailbox moves, the user's mailbox will be unavailable.

    After the mailbox move request has a status of "In Progress," the priority no longer matters and a new move request won’t be processed until an existing In Progress move request is completed, even if the new move request has a higher priority.

Planning   As previously mentioned, because Exchange 2003 users lose access during a hybrid migration, Exchange 2003 customers are usually more concerned about when to schedule migrations and how long they will take.

When planning how many mailboxes to migrate during a specific time period, consider the following:

  • Include the amount of time the move request waits in the queue. Use the following to calculate this:

    (total number of mailboxes to migrate) = ((total time) – (average queue time)) * (migration throughput)

    where the migration throughput equals the total number of mailboxes that can be migrated per hour.

    For example, assume you have a six-hour window to migrate mailboxes. If the average queue time is one hour and you have a migration throughput of 100 mailboxes per hour, you can migrate 500 mailboxes in the six-hour time frame: 500 = (6 – 1) * 100.

  • Start the migration sooner than initially planned to mitigate time in the queue. When mailboxes are queued, Exchange 2003 users can still access their mailboxes.

Determine queue time   The queue time is always changing because Microsoft doesn’t manage customers' migration schedules.

To determine the potential queue time, a customer can try to schedule a test move several hours before the actual migration starts. Then, based on the observed amount of time the request is in the queue, the customer can better estimate when to start the migration and how many mailboxes can be moved in a specific period of time.

For example, say a test migration was completed four hours before the start of a planned migration. The customer determines the queue time for the test migration was about one hour. Then, the customer should consider starting the migration one hour earlier than originally planned to help ensure there is enough time to complete all migrations.

Third-party tools are mostly used in migration scenarios that don't involve Exchange, such as those from Google Mail, IBM Lotus Domino, and Novell GroupWise. This section focuses on the migration protocols used by third-party migration tools, rather than on the actual products and migration tools. The following table provides a list of factors that apply to third-party tools for Office 365 migration scenarios.

 

Checklist Description Best practices

System performance

Data extraction is an intensive task. The source system must have sufficient resources, such as CPU time and memory, to provide optimal migration performance. During migration, the source system is often close to full capacity in terms of the regular end-user workload. If system resources are inadequate, the additional workload that results from migration can affect end users.

Monitor system performance during a pilot migration test. If the system is busy, we recommend avoiding an aggressive migration schedule for the specific system because of potential migration slowness and service availability issues. If possible, enhance the source system performance by adding hardware resources or reducing the load on the system by moving tasks and users to other servers that aren’t involved in the migration.

For more information, see:

When migrating from an on-premises Exchange organization where there are multiple mailbox servers, we recommend that you create a migration user list that’s evenly distributed across multiple mailbox servers. Based on individual server performance, the list can be further fine-tuned to maximize throughput.

For example, if server A has 50 percent more resource availability than server B, it is reasonable to have 50 percent more users from server A in the same migration batch. A similar practice can be applied to other source systems.

Perform migration when the system has maximum resource availability, such as after hours or on weekends and holidays.

Back-end tasks

Other back-end tasks usually run during migration time. Because it’s a best practice to perform migration after business hours, it’s common that migrations conflict with other maintenance tasks running on your on-premises servers, such as data backup.

Review other system tasks that are running during migration. We recommend that you create a clean time window just for data migration, when there are no other resource-heavy tasks.

For Microsoft Exchange on-premises customers, the common tasks are backup solutions. For more information, see Exchange Store Maintenance.

Throttling policy

It’s a common practice to protect email systems with a throttling policy, which sets a specific limit on how fast and how much data can be extracted from the system within a certain amount of time and by using a specific migration method.

Verify what throttling policy is deployed for your email system. For example, Google Mail limits how much data can be extracted in a certain time period.

Depending on the version, Microsoft Exchange has policies that restrict IMAP access to the on-premises mail server (used by IMAP migrations) and RPC over HTTP access (used by cutover Exchange migrations and staged Exchange migrations).

For more information about IMAP throttling, see Migrate Email from an IMAP Server to Exchange Online Mailboxes

For more information about RPC over HTTP throttling, see:

For more information about how to configure EWS throttling, see Exchange 2010: Understanding Client Throttling Policies.

Most third-party tools for Office 365 migrations are client initiated and push data to Office 365. These tools typically require a migration server. Factors such as system performance, back-end tasks, and throttling policies for the source servers apply to these migration servers.

NoteNote:
Some third-party migration solutions are hosted on the Internet as cloud-based services and don’t require an on-premises migration server.

Solution and practice

To improve migration performance when using a migration server, apply the same best practices as the ones described in the Data Source Server section.

For third-party migration tools, the most common protocols used are Exchange Web Service and RPC over HTTP.

Exchange Web Service

Exchange Web Service (EWS) is the recommended protocol to use for migrating to Office 365 because it supports large data batches and has better service-oriented throttling. In Office 365, when used in impersonation mode, migrations using EWS don’t consume the user’s budgeted amount of Office 365 EWS resources, consuming instead a copy of the budgeted resources:

  • All EWS impersonating calls made by the same administrator account are calculated separately from the budget applied to this administrator account.

  • For each impersonation session, a shadow copy of the actual user’s budget is created. All migrations for this particular session will consume this shadow copy.

  • Throttling under impersonation is isolated to each user migration session.

Best practices

  • Migration performance for customers using third-party migration tools that use EWA impersonation competes with EWS–based migrations and service resource usage by other tenants. Therefore, migration performance will vary.

  • Whenever possible, customers should use third-party migration tools that use EWS impersonation because it’s usually faster and more efficient than using client protocols, such as RPC over HTTP.

RPC over HTTP

Many traditional migration solutions use the RPC over HTTP protocol. This method is completely based on a client access model such as that of Outlook, and scalability and performance are limited because the Office 365 service throttles access on the assumption that usage is by a user instead of by an application.

Best practices

  • For migration tools that use RPC over HTTP, it’s a common practice to increase migration throughput by adding more migration servers and using multiple Office 365 administrative user accounts. This practice can gain data injection parallelism and achieve higher data throughput because each administrative user is subject to O365 user throttling. We have received reports that many enterprise customers had to set up more than 40 migration servers to obtain 20–30 GB/hour of migration throughput.

  • In a migration tool development phase, it’s critical to consider the number of RPC operations needed to migrate a message. To illustrate this, we have collected logs captured by Office 365 services for two third-party migration solutions (developed by third-party companies) used by customers to migrate mailboxes to Office 365. We compared two migration solutions developed by third-party companies. We compared the migration of two mailboxes for each migration solution, and we also compared them to uploading a PST file in Microsoft Outlook. Here are the results.

     

    Method Mailbox size Item count Time to migrate Total RPC transactions Average client latency (ms) AvgCasRPCProcessingTime (ms)

    Solution A (mailbox 1)

    376.9 MB

    4,115

    4:24:33

    132,040

    48.4395

    18.0807

    Solution A (mailbox 2)

    249.3 MB

    12,779

    10:50:50

    423,188

    44.1678

    4.8444

    Solution B (mailbox 1)

    618.1 MB

    4,322

    1:54:58

    12,196

    37.2931

    8.3441

    Solution B (mailbox 2)

    56.7 MB

    2,748

    0:47:08

    5,806

    42.1930

    7.4439

    Outlook

    201.9MB

    3,297

    0:29:47

    15,775

    36.9987

    5.6447

    Note that the client and service process times are similar, but solution A takes a lot more RPC operations to migrate data. Because each operation consumes client-latency time and server-process time, solution A is much slower to migrate the same amount of data compared to Solution B and Outlook.

Best practice

For third-party migration solutions that use the RPC over HTTP protocol, here’s a good way to measure potential migration performance:

  1. From the migration server, connect to the Office 365 mailbox with Microsoft Outlook by using RPC over HTTP; ensure that you aren’t connecting by using cache mode.

  2. Import a large PST file with sample data to the Exchange Online mailbox.

  3. Measure migration performance by timing how long it takes to upload the PST file. The migration throughput should be similar to what customers can get from a third-party migration tool that uses RPC over HTTP, given no other constraints. There’s overhead during an actual migration, so the throughput might be slightly different.

Office 365 resource health-base throttling affects migrations using third-party migration tools. See the Office 365 resource health-based throttling section above for more details.

 
Was this page helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2015 Microsoft