Add Allow Rules and Add Disallow Rules Dialog Boxes

Updated: October 21, 2009

Applies To: Windows 7, Windows Server 2008, Windows Server 2008 R2, Windows Vista

Use the Add Allow Rules dialog box to add “allow” rules to the Robots.txt file, and the Add Disallow Rules dialog box to add “disallow” rules to the Robots.txt file. An allow rule means that you want to instruct the search engines that you specify to index the specified path; a disallow rule means that you do not want the search engines that you specify to index the specified path.

 

Element Name Description

Robot (User Agent)

Specify a search engine name, or use “*” to specify all search engines. The default is “*”.

URL Structure

Use the drop-down list to specify how the URL structure of your Web site should be viewed. Select Physical Location to let the folder structure of your Web site determine the hierarchy. Choose From Site Analysis (<SiteAnalysisName>) to obtain the structure from an existing analysis. Choose Run New Site Analysis to open the New Analysis dialog box. For more information about the New Analysis dialog box, see New Analysis and Re-run Report Dialog Boxes.

URL Paths

Select the check boxes of the URL paths that you want to allow or disallow. You can expand or collapse the members of the tree by clicking the plus or minus signs next to the folders.

Disallowed URL Paths

Displays the paths that you allow or disallow when you select or clear paths in the URL Paths pane.

See Also

Community Additions

ADD
Show: