Chapter 11 - Using keyword filtering

Applies to: Microsoft Antigen

Keyword filtering identifies unwanted e-mail messages by analyzing the contents of the message body as it is transported by the SMTP Scan Job. By creating keyword lists, you can filter messages based on a variety of words, phrases, and sentences.

Creating new keyword lists

Antigen for SMTP Gateways comes with filter lists for profanity, racial discrimination, sexual discrimination, and spam. You need to modify these lists as needed to suit the needs of your organization. You can also create additional customized keyword lists as needed to organize your keyword filters.

To create a new keyword list

1. Click the Filter Lists icon in the FILTERING section of the Shuttle Navigator.

2. Select Keywords in the List Types field at the top of the work pane.

3. Click Add in the List Names work pane.

4. Type a name for the new list, and then press ENTER. The empty list appears in the List Names section.

5. With the new list name selected, click Edit. The Edit Filter List dialog box appears. Use it to add content to your filter list.

6. Click the Add button in the Include In Filter section.

7. Type a word or phrase to be included in the filter list. Press ENTER when you have finished typing. You can have as many words or phrases as you want, but each must be entered separately.

   The Exclude from Filter field is used to enter keywords or phrases that should never be included on the keyword list. This prevents those words and phrases from being added accidentally when importing a list from a text file. For more information on importing files, see Importing new items into a filter list.

8. When you are finished adding items, click OK. The list of words that you just entered appears, alphabetically, in the pane next to List Names.

9. Click Save.

Keyword lists must be enabled before you can use them. Each keyword list must be enabled separately.

To enable keyword lists

1. Select Keyword in the FILTERING shuttle. The Keyword Filtering pane appears on the right.

2. Select Message Body in the Keyword Fields box.

3. Select a Filter List to use (profanity, racial discrimination, sexual discrimination, spam, or a customized list).

4. Using the Filter field, set the filter to Enabled.

5. Set the Action as described in Action.

6. Click the General tab and select whether you would like to Send Notifications or Quarantine identified files. Also indicate what combination of Inbound, Outbound, and Internal mail should be scanned.

7. Click the Identify tab and indicate whether the filter should look in the subject line, the message header, or both.

8. Indicate the Minimum Unique Keyword Hits. This setting lets you specify how many unique keywords must be matched for the action to be taken. The default is one (1). For example, you have set the minimum unique keyword hits value to 3. The word "wonderful," which is in the list, appears three times. However, no other word in the list appears at all. The keyword filter has not been matched, because only one term in the list was matched.

9. Click Save.

Action

You need to select the action that Antigen should take upon detecting a match to your filter criteria.

The action choices are described in the following table.

About keyword list syntax rules

The following are the syntax rules for a keyword filter list. Be careful to use the appropriate syntax because Antigen does not perform validation. If the filtering results are not what you are expecting, it is recommended that you double-check your syntax.

• Each item (line of text) is considered a search query.
• Queries use the OR operator. It is considered to be a positive detection if any entry is a match.
• Queries are comprised of operands (keywords), which are text tokens or a string of text tokens, such as:
  • apple (means that the text contains “apple”)
  • apple juice (means that the text contains “apple juice”)
  • get rich quick (means that the text contains “get rich quick”)
• Queries may also contain operators that precede or separate operands in an expression.
• An expression may be comprised of a single operand, an operand preceded by the _NOT_ or _HAS[#]OF_ operators, or two operands joined by the _AND_, _ANDNOT_, or _WITHIN[#]OF_ operators.
  The following logical operators are supported in expressions. There must be a space between an operator and an operand (or another operator), represented in the examples by the • character:
  • _AND_ (logical AND). For example, apples•_AND_•oranges. A filter such as this would be matched if the text contains both “apples” and “oranges”.
  • _NOT_ (negation). For example, _NOT_•oranges. A filter such as this would be matched if the text does not contain “oranges”.
  • _ANDNOT_ (logical AND negation). For example, apples•_ANDNOT_•oranges. A filter such as this would be matched if the text contains “apples” but does not contain “oranges”. _ANDNOT_ is functionally equivalent to _AND_•_NOT_.
  • _HAS[#]OF_ (frequency). Specifies the minimum number of times that the text must appear in order for the query to be considered true. For example, _HAS[4]OF_•get rich quick. If the phrase "get rich quick" is found in the text four or more times, this query is true. This operator implicitly has a default value of 1 when it is not specified.
  • _WITHIN[#]OF_ (proximity). If the two terms are within a specified number of words before or after each other, there is a match. For example, free•_WITHIN[10]OF_•offer. If "free" appears within 10 words before or after "offer", this query is true.
    Multiple operators are permitted in a single query. The precedence of the operators is (from highest to lowest):
  • _WITHIN[#]OF_
  • _HAS[#]OF_
  • _NOT_, _AND_, and _ANDNOT_ (these are at the same precedence level because they are used in conjunction when part of an expression)
    This precedence cannot be overridden with parentheses. Other considerations are:
  • The logical operators must be entered in uppercase letters.
  • Phrases may be used as keywords. For example, apple juice or get rich quick. Quotation marks are not used.
  • Multiple blank spaces (blank characters, line feed characters, carriage return characters, horizontal tabs, and vertical tabs) are treated as one blank space for matching purposes. For example, A••••B is treated as A•B and matches the phrase A•B.
  • In HTML-encoded message texts, punctuation (any non-alphanumeric character) is treated as a word separator similar to blank spaces. Therefore, words surrounded by HTML tags can be properly identified by the filter. However, note that the filter '<html>' will match '<html>', but not 'html'.

Examples (the • character represents a space):

• apples•_AND_•oranges•_AND_•lemons•_WITHIN[50]OF_•juice
  This expression means that “apples”, “oranges”, and “lemons” all appear at least once, and that “lemons” is within 50 words of “juice”.
• confidential•_WITHIN[10]OF_•project•_AND_•banana•_WITHIN[25]OF_•shake
  This expression means that “confidential” is within 10 words of “project”, and that “banana” is within 25 words of “shake”.
• _HAS[2]OF_•get rich•_WITHIN[20]OF_•quick
  This expression means that “get rich” appears at least 2 times within 20 words of “quick”.

About case-sensitive filtering

The General Option setting, Case Sensitive Keyword Filtering, causes Antigen for SMTP Gateways to use case-sensitive comparisons for all keyword filters. By default, all comparisons are not case-sensitive. For more information, see "General Options" in Chapter 4 - Antigen Administrator.

Filtering e-mail messages that automatically load HTML images

To filter e-mail messages that automatically load HTML images from a Web server, add the following items to a keyword filter list:

• img _WITHIN[6]OF_ src="http"
• img _WITHIN[6]OF_ src='http'

These filters will identify instances of the text "img" that occur within six words of the following text: src="http"

If e-mail messages that contain HTML images are not filtered after you add these filters to the keyword list, you may want to examine the source code of the e-mail messages to see how these e-mail messages identify images. Then, you can create additional customized filters.

Creating allowed senders lists

Antigen for SMTP Gateways provides allowed senders list functionality so that administrators can maintain lists of safe e-mail addresses or e-mail domains that are not subjected to filtering or spam scanning. Antigen for SMTP Gateways checks the sender address or domain against the allowed senders list. If the e-mail address or domain appears on the allowed senders list, Antigen for SMTP Gateways bypasses all filtering that has been enabled for the list.

To create an allowed senders list

1. Click the Filter Lists icon in the FILTERING section of the Shuttle Navigator.

2. In the List Types work pane, select Allowed Senders.

3. In the List Names work pane, click Add.

4. Type a name for the new list and then press ENTER. The empty list appears in the List Names work pane.

5. With the new list name selected, click Edit. The Edit Filter List dialog box appears. Use it to enter e-mail addresses or e-mail domains to include in the allowed senders list.

6. In the Include In Filter section, click Add.

7. Type an e-mail address or domain to be included in the filter list, and then press ENTER. User addresses should be entered in the format: user@customer.com. E-mail domain names should be entered in the format: *customer.com. You can have as many allowed senders as you want, but each address or domain must be entered separately.

   The Exclude From Filter section is used to enter addresses or domains that should never be included on the allowed senders list. This prevents those addresses and domains from being accidentally added when importing a list from a text file. For more information on importing files, see Importing new items into a filter list.

8. When you have finished adding items, click OK. The list of addresses and domains that you just entered appears, alphabetically, in the pane next to List Names.

9. Click Save.

To enable an allowed senders list

1. Click the Allowed Senders icon in the FILTERING section of the Shuttle Navigator.

2. In the Allowed Senders work pane, select the scan job from the list in the upper pane.

3. In the Sender Lists work pane, select the allowed senders list.

4. Set the List State to Enabled.

5. In the Skip Scanning for section, indicate whether the allowed senders list should apply to Mailhost Filtering, Content Filtering, Keyword Filtering, or File Filtering. If you have installed the Antigen Spam Manager, you can also select or clear Spam Scanning. You can click All Types to select all options. If none of the check boxes are selected, the filter is effectively disabled.

6. Click Save.

Importing new items into a filter list

Filter lists can be created offline in Notepad or a similar text editor, and then imported into the appropriate filter list by using the Antigen Administrator.

To create and import entries into a filter list

1. Create a list and save it as a text file. Place each filter on its own line in the file.

2. Open the Antigen Administrator, and then click Filter Lists on the FILTERING area of the Shuttle Navigator.

3. Select the filter list into which you will be importing data.

4. Click Edit. The Edit Filter List dialog box appears.

5. Click Import. A File Explorer window opens so that you can navigate to the text file you created in step 1.

6. Select the file, and then click Open.

7. The file is imported into the middle pane of the Import List editor so that you can select the entries you want to include in your filter list. Use the <=== button to move all the items into the Include In Filter section, or use the <--- button to move single items. You can use the right-pointing arrows to move items into the Exclude From Filter section.

8. After you have moved all the desired items, click OK.

9. Click Save.