Processing domain name sets and URL sets

  • Article

Applies To: Forefront Threat Management Gateway (TMG)

Processing domain name sets and URL sets

URL sets and domain name sets are among the Toolbox elements you can create and use when configuring Forefront TMG firewall policy rules:

  • URL sets specify one or more URLs grouped together as a set.

  • Domain name sets define one or more domain names as a single set.

Processing URL Sets

  • URL sets are only processed for Web traffic requests, including HTTP, HTTPS or FTP over HTTP. For other protocols, URL sets specified on an access rule are ignored.

  • You cannot specify a URL set as an IP address.

  • Only the host name and path are considered in a request.

  • In the host part of the name, you can use an asterisk (*) wildcard to specify a set of computers. For example: *.contoso.com.

  • In the path part of the name, you can specify the wildcard character only at the end. For example: www.contoso.com/\*. But not: www.contoso.com/\*/sales.

  • Any protocol or port specified in the URL is stripped and ignored. For example, the following URL set is specified: ftp://a.com:25/apath. Requests for https://a.com will be matched. Requests for different ports, for example https://a.com:55, will also be matched, because port and protocol are stripped.

  • If a request includes a question mark (?), everything following it is stripped before matching. URLs with a ? that are included in a URL set are ignored. For example:

  • If URL set https://www.a.com/apath?next=news is specified in a deny rule, a request for https://www.a.com/apath?next=news will be stripped down to https://www.a.com/apath and will be allowed, because it does not match the deny criteria. To block such a request, you would specify https://www.a.com/apath in the URL set.

  • Host and path names are not case-sensitive when matching.

  • For HTTP (or FTP over HTTP), when the URL requested does not have a path, it matches any path - https://a.cin, or a.com is equivalent to https://a.com/\*. For example:

  • With URL set https://a.com, requests for https://a.com/abc are matched, and requests for https://a.com/abc/def are also matched. With URL set https://a.com/a, requests for https://a.com/a are matched, but requests for https://a.com/a/b are not. Requests are not matched to the tree following "a".

  • HTTPS requests are not matched when path is specified. For example:

  • With URL set a.com, HTTPS requests are matched because no path is specified. If the URL set specified a.com/, HTTPS requests are not matched.

Processing Domain Name Sets

  • When a rule is applied to a domain name set, Forefront TMG checks whether the requests matches the exact domain name set specified, including port numbers. For example:

  • With a domain name set that includes fabrikam.put:1111, requests to fabrikam.put will be defined.

  • You cannot specify a domain name set as an IP address.

  • When you specify a domain name as part of a domain name set, you can use an asterisk (*) to specify a set of computers in the domain. For example, to specify all computers in the contoso.com domain, type the domain name as *.contoso.com.

  • If you specify a wildcard asterisk, it can appear only at the start of the domain name and can be specified only once in the name.

  • When you specify a domain name, specify the computer name using the fully qualified domain name (FQDN). For example, computer_name.contoso.com, and not \\computer_name.

  • When you create a domain with a wildcard character, such as *.contoso.com, this only includes host computers at the domain, for example www.contoso.com, ftp.contoso.com. Note that if the domain name points to a host, *.contoso.com will have no effect on the URL https://Contoso.com.

  • We recommend that you enter the domain name as it is returned by DNS. If you specify a dot at the end of a domain name, a request for the domain name (without a dot) may not be matched as required.

  • When matching rules, the domain name is not case-sensitive.

Name Resolution

Rules that include domain name sets and URL sets require name resolution. If there are no rule criteria that prevent rule matching, and the rule may match the request if name resolution is performed, the rule will be subject to name resolution. For example, if the rule contains a URL set but a schedule limitation on the rule prevents matching, the rule is not subject to name resolution. The following types of requests may be marked for name resolution:

  • A Web request specified by name encounters a rule that has an address range specified as the destination criteria (forward lookup)

  • A Web request specified by IP address encounters a rule that has a URL set as the destination criteria (reverse lookup).

The Microsoft Firewall service includes its own Domain Name System (DNS) cache. If the requested IP address or host name resides in this cache, the request is processed without issuing a DNS request. Otherwise, a DNS request is issued. Name resolution provides a host entry, and the rules engine then compares the host entry against the destination criteria of the rule. The rules engine does a string compare against URL sets and domain name set entries.

Note that rules requiring name resolution are evaluated and enforced in accordance with DNS resolution information. If DNS information is not configured correctly or securely, rules may not be applied as required.