Export (0) Print
Expand All

Manage continuous crawls in SharePoint Server 2013

SharePoint 2013
 

Applies to: SharePoint Server 2013

Topic Last Modified: 2013-12-18

Summary: Continuous crawls crawl SharePoint sites frequently to help keep search results fresh. Learn about how to enable and disable continuous crawls, and how to change the frequency interval.

Enable continuous crawls is a crawl schedule option that is an alternative to incremental crawls. This option is new in SharePoint 2013 and applies only to content sources of type SharePoint Sites.

Like incremental crawls, a continuous crawl crawls content that was added, changed, or deleted since the last crawl. Unlike an incremental crawl, which starts at a particular time and repeats regularly at specified times after that, a continuous crawl automatically starts at predefined time intervals. The default interval for continuous crawls is every 15 minutes. Continuous crawls help ensure freshness of search results because the search index is kept up to date as the SharePoint content is crawled so frequently. Thus, continuous crawls are especially useful for crawling SharePoint content that is quickly changing.

A single continuous crawl includes all content sources in a Search service application for which continuous crawls are enabled. Similarly, the continuous crawl interval applies to all content sources in the Search service application for which continuous crawls are enabled.

You cannot run multiple full crawls or multiple incremental crawls for the same content source at the same time. However, multiple continuous crawls can run at the same time. Therefore, even if one continuous crawl is processing a large content update, another continuous crawl can start at the predefined time interval and crawl other updates. Continuous crawls of a particular content repository can also occur while a full or incremental crawl is in progress for the same repository.

A continuous crawl does not process or retry items that return errors more than three times. A "clean-up" incremental crawl automatically runs every four hours for content sources that have continuous crawl enabled to re-crawl any items that repeatedly return errors. This incremental crawl will try to crawl the item again and then will postpone retries if the error persists.

You can set incremental crawl times on the Search_Service_Application_Name: Add/Edit Content Source page, but you can change the frequency interval for continuous crawls only by using Windows PowerShell.

In this article:

NoteNote:
Because SharePoint 2013 runs as websites in Internet Information Services (IIS), administrators and users depend on the accessibility features that browsers provide. SharePoint 2013 supports the accessibility features of supported browsers. For more information, see the following resources:

To enable continuous crawls for an existing content source
  1. Verify that the user account that is performing this procedure is an administrator for the Search service application.

  2. In Central Administration, in the Application Management section, click Manage service applications.

  3. Click the Search service application.

  4. On the Search_Service_Application_Name: Search Administration page, in the Quick Launch, under Crawling, click Content Sources.

  5. On the Search_Service_Application_Name: Manage Content Sources page, click the SharePoint content source for which you want to enable continuous crawl.

  6. In the Crawl Schedules section, select Enable Continuous Crawls.

  7. Click OK.

  8. Verification: On the Search_Service_Application_Name: Manage Content Sources page, verify that the Status column has the status Crawling Continuous.

To enable continuous crawls for a new content source
  1. Verify that the user account that is performing this procedure is an administrator for the Search service application.

  2. In Central Administration, in the Application Management section, click Manage service applications.

  3. Click the Search service application.

  4. On the Search_Service_Application_Name: Search Administration page, in the Quick Launch, under Crawling, click Content Sources.

  5. On the Search_Service_Application_Name: Manage Content Sources page, click New Content Source.

  6. Create a content source of the type SharePoint Sites.

    1. In the Name section, type a name in the Name field.

    2. In the Content Source Type section, select SharePoint Sites.

    3. In the Start Addresses section, type the start address or addresses.

    4. In the Crawl Settings section, select the crawling behavior for all start addresses.

    5. In the Crawl Schedules section, select Enable Continuous Crawls.

  7. Click OK.

  8. Verification: On the Search_Service_Application_Name: Manage Content Sources page, verify that the newly added content source appears and that the Status column has the status Crawling Continuous.

To disable continuous crawls for a content source
  1. Verify that the user account that is performing this procedure is an administrator for the Search service application.

  2. In Central Administration, in the Application Management section, click Manage service applications.

  3. Click the Search service application.

  4. On the Search_Service_Application_Name: Search Administration page, in the Quick Launch, under Crawling, click Content Sources.

  5. On the Search_Service_Application_Name: Manage Content Sources page, click the SharePoint content source for which you want to disable continuous crawls.

  6. In the Crawl Schedules section, clear Enable Incremental Crawls. This disables continuous crawls.

  7. To confirm that you want to disable continuous crawls, click OK.

  8. Optional: click Edit schedule to change the schedule for incremental crawls, and then click OK.

  9. On the Search_Service_Application_Name: Edit Content Source page, click OK.

  10. Verification: On the Search_Service_Application_Name: Manage Content Sources page, verify that the Status column has changed to Idle. This might take some time, because all URLs that remain in the crawl queue are still crawled after you disable continuous crawls.


To disable continuous crawls for all content sources
  1. Verify that the user account that performs this procedure is an administrator for the Search service application.

  2. Start a SharePoint 2013 Management Shell on a server in the farm.

    • For Windows Server 2008 R2:

      On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, then right-click SharePoint 2013 Management Shell and then click Run as administrator.

    • For Windows Server 2012:

      1. On the Start screen, right-click SharePoint 2013 Management Shell and then click Run as administrator.

        If SharePoint 2013 Management Shell does not appear on the Start screen:

      2. Right-click Computer, click All apps, right-click SharePoint 2013 Management Shell, and then click Run as administrator.

    For more information about how to use Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.

  3. At the Windows PowerShell command prompt, type the following commands:

    $SSA =  Get-SPEnterpriseSearchServiceApplication
    $SPContentSources = $SSA | Get-SPEnterpriseSearchCrawlContentSource | WHERE {$_.Type -eq "SharePoint"} 
    foreach ($cs in $SPContentSources) 
    { 
      $cs.EnableContinuousCrawls = $false 
      $cs.Update() 
    }
    
  4. Verification: On the Search_Service_Application_Name: Manage Content Sources page, verify that the Status column has changed to Idle for all content sources. This might take some time, because all URLs that remain in the crawl queue are still crawled after you disable continuous crawls.


To change the continuous crawl interval
  1. Verify that the user account that is performing this procedure is a member of the Farm Administrators group.

  2. Start a SharePoint 2013 Management Shell.

    • For Windows Server 2008 R2:

      On the Start menu, click All Programs, click Microsoft SharePoint 2013 Products, right-click SharePoint 2013 Management Shell and then click Run as administrator.

    • For Windows Server 2012:

      1. On the Start screen, right-click SharePoint 2013 Management Shell, and then click Run as administrator.

        If SharePoint 2013 Management Shell does not appear on the Start screen:

      2. Right-click Computer, click All apps, right-click SharePoint 2013 Management Shell and then click Run as administrator.

    For more information about how to interact with Windows Server 2012, see Common Management Tasks and Navigation in Windows Server 2012.

  3. At the Windows PowerShell command prompt, type the following commands:

    $ssa = Get-SPEnterpriseSearchServiceApplication
    $ssa.SetProperty("ContinuousCrawlInterval",n)
    

    Where:

    • n is the regular interval in minutes at which you want to continuous crawls to start. The default interval is every 15 minutes. The shortest interval that you can set is 1 minute.

      NoteNote:
      If you reduce the interval, you increase the load on SharePoint and the crawler. Make sure that you plan and scale out for this increased consumption of resources accordingly.

Was this page helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft