URL filtering common issues
Updated: February 1, 2011
Applies To: Forefront Threat Management Gateway (TMG)
This topic contains troubleshooting information for the following issues you may encounter when URL filtering is enabled in your Forefront TMG deployment:
Slow user access to requested sites
All blocked sites are categorized as unknown
A site’s categorization is incorrect
Access to blocked sites is allowed when using Web translation services
Slow user access to requested sites
In some cases, user access to requested sites is slow.
Cause
This might be caused by slow responses from Microsoft Reputation Services (MRS) due to network or performance issues. It only affects requests for sites in which a site’s URL filtering categorization is not cached on Forefront TMG; requests for sites in which the categorization is cached on Forefront TMG, and sites in which Forefront TMG applies a URL override, are unaffected.
Solution
To determine whether slow responses from MRS are causing this issue, check the Forefront TMG performance counters, for example:
URL Filtering - Avg Categorization Duration.
URL Filtering - % Slow Categorizations from Server.
URL Filtering - % Very Slow Categorizations from Server.
For information, see URL filtering performance counters and Monitoring performance counters.
All blocked sites are categorized as unknown
When users attempt to access blocked sites, the denial notification they receive specifies the site category as unknown, for all sites.
Cause
This might happen if Forefront TMG fails to query the MRS server for the site’s categorization, for example, if the MRS server is paused or is down.
In the Forefront TMG Management console, the following alerts are displayed:
URL Categorization Server Unavailable—Displayed following a number of failures to connect to the MRS server.
URL Categorization Server Paused—Displayed when the MRS server is paused.
For information, see Monitoring alerts.
Solution
To determine the cause of this issue, query the Forefront TMG logs for URL Categorization Reason. For information, see Web proxy log fields and Configuring Forefront TMG logs.
To troubleshoot the failure to query the MRS server, see URL filtering troubleshooting flow.
A site’s categorization is incorrect
Users attempting to access a site are blocked, and the category that is specified in the denial notification is incorrect.
Cause
Incorrect categorization of a site can be caused by one of the following reasons:
Incorrect categorization by MRS.
A URL may be defined in MRS by more than one category, whereas Forefront TMG assigns a single category to each URL according to a predefined precedence list. The category that Forefront TMG assigned to the URL is incorrect.
Solution
To change the categorization of a site:
In the Forefront TMG Management console, query the URL’s category. For information, see Looking up a URL category.
The query results indicate the source of the categorization. If the site was categorized by MRS, suggest alternate URL categorizations at Microsoft Reputation Services Feedback and Error Reporting (https://go.microsoft.com/fwlink/?LinkId=181927).
You can also assign a different category to the site in Forefront TMG. For information, see Overriding URL categorization.
Access to blocked sites is allowed when using Web translation services
When users use Web translation services, they are allowed access to sites that should be blocked according their categorization.
Cause
Because Web translation services may retrieve the Web page on behalf of Forefront TMG, the URL filtering mechanism processes the URL of the translation site, not that of the requested site.
Solution
To resolve this issue, do one of the following:
Prevent end users from using Web translation services.
Discover the URL of the request for Web translation, and block this URL. Note that you need to distinguish between requests for Web page translation from requests for text-block translation. For example, when you use Microsoft Translator, you can distinguish between the two requests, as follows:
A request tor text-block translation appears to Forefront TMG as https://www.microsofttranslator.com/Default.aspx.
A request for Web page translation appears to Forefront TMG as https://www.microsofttranslator.com/BV.aspx?ref=Internal\&a=http%3a%2f%2fwww.it-training-grote.de%2f.
In this case, the URL you must block in order to prevent this issue is *.microsofttranslator.com/BV.aspx. For information, see Introduction to configuring web access.