Add, edit or remove a FAST Web crawler configuration

  • Article

 

Applies to: FAST Search Server 2010

Add a configuration

  1. Copy the configuration template CrawlerConfigTemplate-Simple.xml that is located in the FAST Search Server 2010 for SharePoint installation etc folder, typically C:\FASTSearch\etc. Give the copied template a new unique name, for example MyCrawlerConfig.xml.

  2. Configure the parameter settings for your environment in a text or XML editor of your choice. Refer to Web Crawler XML configuration for a description of all the available FAST Search Web crawler parameters and their settings.

    Warning

    Each content collection should have a separate crawl configuration. The default content collection is sp. If you have multiple content collections, create a new crawl configuration for each content collection and indicate the applicable collection name in the crawl configuration file. Add the content collection name in the parameter DomainSpec name.

  3. Save the configuration file, this will make it easier to edit the configuration at a later stage. Make sure the file does not include a Byte-Order Mark (BOM) / UTF-8 signature.

  4. The FAST Search Web crawler configuration is now ready to use. The FAST Search Web crawler needs to run in order for the configuration to be activated. Add the crawl configuration to the FAST Search Web crawler by using the command crawleradmin -f <Path to XML file>, where <Path to XML file> is the path to your FAST Search Web crawler configuration file, for example C:\FASTSearch\etc\MyCrawlerConfig.xml.

Edit a configuration

  1. Open the configuration file that needs to be edited.

  2. Make the necessary edits in a text or XML editor of your choice.

  3. The FAST Search Web crawler needs to run in order to activate the changes in the FAST Search Web crawler configuration. Add the edited crawl configuration to the FAST Search Web crawler by using the command crawleradmin -f <Path to XML file>, where <Path to XML file> is the path to your FAST Search Web crawler configuration file, for example C:\FASTSearch\etc\MyCrawlerConfig.xml.

Remove a configuration

This procedure removes a particular FAST Search Web crawler crawl configuration from all servers that run one or more FAST Search Web crawler components. The FAST Search Web crawler configuration file(s) will not be deleted in this process and can be reused or removed later. This procedure does not remove any data from the content index. Refer to the topic Reset the content index (FAST Search Web crawler) for instructions on how to remove items from the content index.

  1. At the command prompt, navigate to <FASTSearchFolder>\bin, where <FASTSearchFolder> is the path of the folder where you have installed FAST Search Server 2010 for SharePoint, for example C:\FASTSearch.

  2. At the command prompt, type the following command:

    crawleradmin –d <MyCollection>

    Where <MyCollection> is the file name of the crawler configuration that you want to remove.

  3. Double check that you have entered the correct name of the crawl configuration that you are about to remove and press ENTER. Press ENTER again when you are prompted to confirm that you want to delete the configuration.

  4. Wait for the command to finish. This may take some time.

See Also

Reference

crawleradmin.exe reference
Clear-FASTSearchContentCollection

Concepts

Web Crawler XML configuration reference
Crawling Web content with the FAST Search Web crawler
Start, suspend, resume, or stop a crawl process (FAST Search Web crawler)
Reset the content index (FAST Search Web crawler)