Add Allow Rules and Add Disallow Rules Dialog Boxes
Applies To: Windows 7, Windows Server 2008, Windows Server 2008 R2, Windows Vista
Use the Add Allow Rules dialog box to add “allow” rules to the Robots.txt file, and the Add Disallow Rules dialog box to add “disallow” rules to the Robots.txt file. An allow rule means that you want to instruct the search engines that you specify to index the specified path; a disallow rule means that you do not want the search engines that you specify to index the specified path.
UI Elements
| Element Name | Description |
|---|---|
Robot (User Agent) |
Specify a search engine name, or use “*” to specify all search engines. The default is “*”. |
URL Structure |
Use the drop-down list to specify how the URL structure of your Web site should be viewed. Select Physical Location to let the folder structure of your Web site determine the hierarchy. Choose From Site Analysis (<SiteAnalysisName>) to obtain the structure from an existing analysis. Choose Run New Site Analysis to open the New Analysis dialog box. For more information about the New Analysis dialog box, see New Analysis and Re-run Report Dialog Boxes. |
URL Paths |
Select the check boxes of the URL paths that you want to allow or disallow. You can expand or collapse the members of the tree by clicking the plus or minus signs next to the folders. |
Disallowed URL Paths |
Displays the paths that you allow or disallow when you select or clear paths in the URL Paths pane. |