You can create a URL set, and then use it in access rules to allow or deny access to websites specified in the set. When ISA Server processes a rule that applies to a URL set, the URL set element of the rule is only processed for Web traffic requests (HTTP, HTTPS, or FTP over HTTP). If a client request uses another protocol (Firewall traffic), ISA Server ignores the URL set when processing the rule. For example, if a rule has both a Computer set and a URL set specified as destination criteria, only the Computer set will be evaluated in the rule. The URL set will be ignored.

Note the following when creating URL sets:

• You can specify one or more URLs in URL format:
  
  • <protocol>://<host>:<port>/<path>
    
• In the host part of the name, you can use a wildcard asterisk (*) to specify a set in computers. For example to specify all computers in the Microsoft.com domain, specify *.microsoft.com.
  
• In the path part of the name, you can specify a wildcard asterisk as part of the path, but only at the end. For example:
  
  • www.microsoft.com/* is acceptable.
    
  • www.microsoft.com/*/sales is not acceptable.
    
• You cannot specify a URL set as an IP address.
  

Note the following behavior in matching requests with rules containing URL sets:

• Only the host name and path are considered in a request.
  
  • The protocol part of the URL is stripped from requests and ignored.
    
  • Any port number specified is stripped from requests and ignored.
    
• If a request includes a question mark (?), the question mark and everything following it is stripped from the request before matching.
  
• When matching, the host and path names are not case-sensitive.
  
• For HTTP and FTP over HTTP, when the URL is specified in a request without a path, it will match any path. In other words, http://a.com, or "a.com" is equivalent to http://a.com/*.
  
• For HTTPS traffic, URL sets are only processed if the URL does not have a path specified. For example, http://a.com or "a.com". If the URL has a path specified (even "/"), it is ignored for HTTPS traffic.
  

Some URL set mapping examples are as follows:

For URL set entry:

• ftp://a.com:25/apath
  
  • Requests for http://a.com will be matched, as will requests for http://a.com:55, because protocol and port are stripped.
    
• For URL set entry:
  
  • http://a.com
    
• Requests for http://a.com/abc will be matched, as will requests for http://a.com/abc/def. In other words, http://a.com is the equivalent of http://a.com/*. The exception is for HTTPS requests, which will not be processed because a path is specified.
  
• For URL set entry:
  
  • http://a.com/a
    
• Requests for http://a.com/a will be matched. But requests for http://a.com/a/b will not be matched. In such an entry, requests are not matched to the tree following "a".
  
• For URL set entry:
  
  • http://www.a.com/apath?next=news
    
• The question mark and everything following are stripped from requests, so if this URL set was specified in a deny rule and a request arrived for http://www.a.com/apath?next=news, the request would be stripped down to http://www.a.com/apath, and would be allowed because it does not match the URL set specified in the deny rule. To block such a request, you should specify http://www.a.com/apath in the URL set.
  
• For URL set entry:
  
  • "a.com", HTTPS requests will be matched, because no path is specified.
    
• For URL set entry:
  
  • "b.com/", HTTPS requests will not be matched, because a path ("/") is specified.
    

Domain sets gather together one or more domain names as a single set, for use in your firewall policy.

Note the following:

• You cannot specify a domain name set as an IP address.
  
• When you specify a domain name as part of a domain name set, you can use an asterisk (*) to specify a set of computers in the domain. For example, to specify all computers in the microsoft.com domain, type the domain name as *.microsoft.com.
  
• If you specify a wildcard asterisk, it can appear only at the start of the domain name, and can be specified only once in the name.
  
• When you specify a domain name, specify the computer name using the fully qualified domain name (FQDN). For example, computer_name.microsoft.com, and not \\computer_name.
  
• When you create a domain with a wildcard character, such as *.microsoft.com, this only includes host computers at the domain, for example www.microsoft.com, ftp.microsoft.com. Note that if the domain name points to a host, *.microsoft.com will have no effect on the URL http://Microsoft.com.
  
• We recommend that you enter the domain name as it is returned by DNS. If you specify a dot at the end of a domain name, a request for the domain name (without a dot) may not be matched as required.
  
• When matching rules, the domain name is not case-sensitive.
  

ISA Server determines whether requests should be allowed or denied in accordance with access policy. The ISA Server rules engine attempts to match access rules, and then routing rules, with requests. Rules that include domain name sets and URL sets require name resolution, among others. If there are no rule criteria which prevent rule matching, and the rule may match the request if name resolution is performed, the rule will be subject to name resolution. In other words, if the rule contains a URL set, but a schedule limitation on the rule prevents matching, the rule is not subject to name resolution. The following types of requests may be marked for name resolution:

• A Web request specified by name encounters a rule that has an address range specified as the destination criteria (forward lookup).
  
• A Web request specified by IP address encounters a rule that has a URL set as the destination criteria (reverse lookup).
  

The Firewall service includes its own DNS cache. If the requested IP address or host name resides in this cache, the request is processed without issuing a DNS request. Otherwise, a DNS request is issued using the Windows® API. Name resolution provides a host entry, and the rules engine then compares the host entry against the destination criteria of the rule. The rules engine does a string compare against URL sets and domain name set entries.

It is important to note that rules requiring name resolution are evaluated and enforced in accordance with DNS resolution information. If DNS information is not configured correctly or securely, rules may not be applied as required.