Adding Lists to Domain Name Sets and URL Sets

  • Article

Microsoft Internet Security and Acceleration (ISA) Server 2004 provides the following two types of rule elements into which you can copy lists of sites that you want to block using an access rule:

  • Domain name sets
  • Uniform Resource Locator (URL) sets

A domain name set is defined by an FPCDomainNameSet collection. The domain name sets defined in an array or on the enterprise level (in Enterprise Edition) are elements of an FPCDomainNameSets collection. Each domain name set in this collection contains a set of domain names that can be applied to cache rules, routing rules, access rules, and system policy rules.

The domain names included in a domain name set may be specified in either of the following formats:

  • Fully qualified domain name (FQDN) (for example, www.northwindtraders.com)
  • Domain Name System (DNS) suffix (for example, *.net)

Domain names specified in other formats may be included, but these are ignored.

A URL set is defined by an FPCURLSet collection. The URL sets defined in an array or on the enterprise level (in Enterprise Edition) are elements of an FPCURLSets collection. Each URL set in this collection contains a set of URLs that can be applied to rules that control HTTP traffic, including cache rules, routing rules, access rules, and system policy rules.

Each URL in a URL set may include a host name and a path. Wildcard characters are allowed. However, URLs containing a question mark (?) that are included in a URL set are ignored. A protocol (HTTP, HTTPS, or FTP) and a port number may be included, but these are ignored.

Host names may be specified in any of the following formats:

Paths may be specified in any of the following formats:

  • Full path (for example, default.htm)
  • Prefix (for example, /pictures/travel/* or /*)

If you have a large list of domain names or URLs in a text file, you can add them to a domain name set or a URL set directly from the text file. This can be accomplished by reading each line of the text file into a string and calling the Add method of the applicable FPCDomainNameSet or FPCURLSet collection to add the string as an element of the collection. Then you can apply the domain name set or URL set to an access rule represented by an FPCPolicyRule object by adding a reference object (an FPCRef object) for it to the collection of reference objects held in the DestinationDomainNameSets or URLSets property of the access rule.

Insert introduction here.

Adding a List of Sites to a Domain Name Set

The Microsoft Visual Basic Scripting Edition (VBScript) code in AddListToDomainNameSet.vbs (listed later in this document) creates a new domain name set with the name specified by the user if it does not exist and imports all the domain names listed in a text file to the domain name set. If the specified domain name set already exists, the domain names in the text file are added without removing the existing domain names. In either case, duplicates are not added.

Usage:CScript AddListToDomainNameSet.vbs FileName ** DNSetName

FileName specifies the name of a text file containing a list of domain names.

DNSetName specifies the name of the domain name set to which the list will be added.

To add a list of sitesto a domain name set

  1. Create an instance of the FPC COM object, which is known as the root object and provides access to the other ISA Server administration COM objects.

  2. Declare an FPCArray object, an FPCDomainNameSets collection, an FPCDomainNameSet object, a FileSystemObject object, and a TextStream object.

  3. Get references to the FPCArray object and the FPCDomainNameSets collection.

  4. Try to retrieve the domain name set specified by the user from the domain name sets collection for the array. If the domain name set does not exist, create it.

  5. Create an instance of the FileSystemObject object, and call the OpenTextFile method on it to retrieve a TextStream object that represents the text file specified by the user.

  6. In a While loop, call the ReadLine method of the TextStream object to read each successive line in the text file. Try to retrieve the domain name on each line from the domain name set specified by the user. If a domain name is not found in the domain name set, add it to the domain name set.

  7. Call the Save method of the domain name sets collection to write the configuration changes to storage.

Script Listing: AddListToDomainNameSet.vbs

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

' Copyright (c) Microsoft Corporation. All rights reserved.

' THIS CODE IS MADE AVAILABLE AS IS, WITHOUT WARRANTY OF ANY KIND. THE ENTIRE

' RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE

' USER. USE AND REDISTRIBUTION OF THIS CODE, WITH OR WITHOUT MODIFICATION, IS

' HEREBY PERMITTED.

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

' This script creates a new domain name set with the name specified by the user

' if it does not exist and imports all the domain names listed in a text file

' to the domain name set. If the specified domain name set already exists, the

' domain names in the text file are added without removing the existing domain

' names. In either case, duplicates are not added.

' This script has minimal error handling.

' Note that the text file must contain a list of domain names with each domain

' name on a separate line.

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

Option Explicit

'Define the constants needed

Const Error_FileNotFound = &H80070002

Const ForReading = 1

Main(WScript.Arguments)

Sub Main(args)

If(args.Count <> 2) Then

Usage

End If

AddListToDomainNameSet args(0), args(1)

End Sub

Sub AddListToDomainNameSet(fileName, dnSetName)

' Create the root object.

Dim root ' The FPCLib.FPC root object

Set root = CreateObject("FPC.Root")

'Declare the other objects needed.

Dim isaArray ' An FPCArray object

Dim dnSets ' An FPCDomainNameSets collection

Dim dnSet ' An FPCDomainNameSet object

Dim fso ' A FileSystemObject object

Dim fileStream ' A TextStream object

Dim textRead ' A string

' Get references to the array object

' and the domain name sets collection.

Set isaArray = root.GetContainingArray()

Set dnSets = isaArray.RuleElements.DomainNameSets

' Retrieve the specified domain name set.

On Error Resume Next

Set dnSet = dnSets.Item(dnSetName)

If Err.Number = Error_FileNotFound Then

WScript.Echo "The " & dnSetName & "domain name set does not exist. " _

& "Creating it ..."

Set dnSet = dnSets.Add(dnSetName)

End If

On Error GoTo 0

Set fso = CreateObject("Scripting.FileSystemObject")

Set fileStream = fso.OpenTextFile(fileName, ForReading)

On Error Resume Next

Do While fileStream.AtEndOfStream <> True

textRead = fileStream.ReadLine

If textRead <> "" Then

Err.Clear

dnSet.Item textRead

If Err.Number = Error_FileNotFound Then

Err.Clear

WScript.Echo "Adding " & textRead

dnSet.Add textRead

End If

End If

Loop

On Error GoTo 0

' Save the changes.

dnSets.Save

WScript.Echo "Done!"

End Sub

Sub Usage()

WScript.Echo "Usage:" & VbCrLf _

& " CScript " & WScript.ScriptName & " FileName DNSetName" & VbCrLf _

& "" & VbCrLf _

& " FileName - Text file containing a list of domain names" & VbCrLf _

& " DNSetName - Domain name set to which the list will be added"

WScript.Quit

End Sub

Adding a List of Sites to a URL Set

The VBScript code in AddUrlsToUrlSet.vbs (listed later in this document) creates a new URL set with the name specified by the user if it does not exist and imports all the URLs listed in a text file to the URL set. If the specified URL set already exists, the URLs in the text file are added without removing the existing URLs. In either case, duplicates are not added.

Usage:CScript AddUrlsToUrlSet.vbs FileName UrlSetName

FileName specifies the name of a text file containing a list of URLs.

UrlSetName specifies the name of the URL set to which the list will be added.

To add a list of sites to a URL set

  1. Create an instance of the FPC COM object, which is known as the root object and provides access to the other ISA Server administration COM objects.

  2. Declare an FPCArray object, an FPCURLSets collection, an FPCURLSet object, a FileSystemObject object, and a TextStream object.

  3. Get references to the FPCArray object and the FPCURLSets collection.

  4. Try to retrieve the URL set specified by the user from the URL sets collection for the array. If the URL set does not exist, create it.

  5. Create an instance of the FileSystemObject object, and call the OpenTextFile method on it to retrieve a TextStream object that represents the text file specified by the user.

  6. In a While loop, call the ReadLine method of the TextStream object to read each successive line in the text file. Try to retrieve the URL on each line from the URL set specified by the user. If a URL is not found in the URL set, add it to the URL set.

  7. Call the Save method of the URL sets collection to write the configuration changes to storage.

Script Listing: AddUrlsToUrlSet.vbs

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

' Copyright (c) Microsoft Corporation. All rights reserved.

' THIS CODE IS MADE AVAILABLE AS IS, WITHOUT WARRANTY OF ANY KIND. THE ENTIRE

' RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE

' USER. USE AND REDISTRIBUTION OF THIS CODE, WITH OR WITHOUT MODIFICATION, IS

' HEREBY PERMITTED.

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

' This script creates a new URL set with the name specified by the user if it

' does not exist and imports all the URLs listed in a text file to the URL

' set. If the specified URL set already exists, the URLs in the text file are

' added without removing the existing URLs. In either case, duplicates are

' not added.

' Note that the text file must contain a list of URLs with each URL

' on a separate line.

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

Option Explicit

'Define the constants needed

Const Error_FileNotFound = &H80070002

Const ForReading = 1

Main(WScript.Arguments)

Sub Main(args)

If(args.Count <> 2) Then

Usage

End If

AddUrlsToUrlSet args(0), args(1)

End Sub

Sub AddUrlsToUrlSet(fileName, urlSetName)

' Create the root object.

Dim root ' The FPCLib.FPC root object

Set root = CreateObject("FPC.Root")

'Declare the other objects needed.

Dim isaArray ' An FPCArray object

Dim urlsets ' An FPCURLSets collection

Dim urlset ' An FPCURLSet object

Dim fso ' A FileSystemObject object

Dim fileStream ' A TextStream object

Dim textRead ' A string

' Get references to the array object

' and the URL sets collection.

Set isaArray = root.GetContainingArray()

Set urlsets = isaArray.RuleElements.URLSets

' Retrieve the specified URL set.

On Error Resume Next

Set urlset = urlsets.Item(urlSetName)

If Err.Number = Error_FileNotFound Then

WScript.Echo "The " & urlSetName & " URL set does not exist. " _

& "Creating it ..."

Set urlset = urlsets.Add(urlSetName)

End If

On Error GoTo 0

Set fso = CreateObject("Scripting.FileSystemObject")

Set fileStream = fso.OpenTextFile(fileName, ForReading)

On Error Resume Next

Do While fileStream.AtEndOfStream <> True

textRead = fileStream.ReadLine

If textRead <> "" Then

Err.Clear

urlset.Item textRead

If Err.Number = Error_FileNotFound Then

Err.Clear

WScript.Echo "Adding " & textRead

urlset.Add textRead

End If

End If

Loop

On Error GoTo 0

' Save the changes.

urlsets.Save

WScript.Echo "Done!"

End Sub

Sub Usage()

WScript.Echo "Usage:" & VbCrLf _

& " CScript " & WScript.ScriptName & " FileName UrlSetName" & VbCrLf _

& "" & VbCrLf _

& " FileName - Text file containing the list of URLs" & VbCrLf _

& " UrlSetName - URL set to which the list of URLs will be added"

WScript.Quit

End Sub