Overview of search architecture in SharePoint Server

APPLIES TO: 2013 2016 2019 Subscription Edition SharePoint in Microsoft 365

The search architecture contains search components and databases. How you structure the search architecture depends on where you intend to use search: for the enterprise or for Internet sites. When building the search architecture, you should take into account considerations such as high availability and fault tolerance, the volume of your content and the estimated amount of page views and queries per second.

For information about search topologies for different use cases: see the technical diagrams Enterprise search architectures for SharePoint Server 2016 and Internet sites search architectures for SharePoint Server 2016.

The following tables show an overview of all the available search components and search databases. For more information about how search components and databases interact, see the Search architectures for SharePoint Server 2016 technical diagram.

Search components

Search databases

The crawl component crawls the content sources. You can crawl lots of content sources, for example file shares, SharePoint Server content, line of business applications and many more. To retrieve information, the crawl component connects to the content sources by invoking the appropriate indexing connector or protocol handler. After retrieving the content, the crawl component passes crawled items to the content processing component.

For more information about crawling content sources, see Plan crawling and federation in SharePoint Server.

The content processing component processes crawled items and sends these items to the index component. The content processing component performs operations such as document parsing and property mapping. It also performs linguistics processing such as language detection and entity extraction. The component transforms crawled items into artifacts that are included in the search index. The content processing component also writes information about links and URLs to the link database.

For more information about content processing, see Plan crawling and federation in SharePoint Server.

The analytics processing component performs two types of analyses: search analytics and usage analytics. This component uses information from these analyses to improve search relevance, create search reports, and generate recommendations and deep links.

• Search analytics is about extracting information, such as links, the number of times an item is clicked, anchor text, data related to people, and metadata, from the link database. This information is important to relevance.

• Usage analytics is about analyzing usage log information received from the front-end via the event store. Usage analytics generates usage and statistics reports.

The results from the analyses are added to the items in the search index. In addition, results from usage analytics are stored in the analytics reporting database.

For more information, see Overview of analytics processing in SharePoint Server.

You can divide the search index into discrete portions, called index partitions. The search index is the aggregation of all index partitions. Each index partition holds one or more index replicas that contain the same information. To achieve fault tolerance and redundancy, create additional index replicas for each index partition and distribute the index replicas over multiple servers.

The index component is the logical representation of an index replica. In the search topology, you have to provision one index component for each index replica.

The index component:

• Receives processed items from the content processing component and writes those items to an index file. Index files are stored on a disk in the server that hosts the index component.

• Receives queries from the query processing component and returns result sets.

For more information about the search schema and the search index, see Overview of the search schema in SharePoint Server.

The query component analyzes and processes queries and results. It performs linguistics processing such as word breaking and stemming. When the query processing component receives a query from the search front-end, it analyzes and processes the query to optimize precision, recall and relevance. The processed query is submitted to the index component. The index component returns a result set based on the processed query to the query processing component, which in turn processes that result set, before returning it to the search front-end.

For more information, see Plan to transform queries and order results in SharePoint Server.

The search administration component runs the system processes for search. This component performs provisioning, which is to add and initialize instances of the other search components.

The crawl database stores tracking information and historical information about crawled items. For example, it stores information about the last crawl time, the last crawl ID and the type of update during the last crawl.

The link database stores information extracted by the content processing component. In addition, it stores information about search clicks; the number of times people click on a search result from the search result page. This information is stored unprocessed, to be analyzed by the analytics processing component.

The analytics reporting database stores the results of usage analytics. In addition, it stores statistics information from the analyses. SharePoint Server uses this information to create Excel reports that show different statistics.

The search administration database stores search configuration data, such as the topology, crawl rules, query rules, and the mappings between crawled and managed properties. It also stores the access control list (ACL) for the crawl component. There can be only one search administration database per search service application.

Manage the search topology in SharePoint Server