Overview of intrusion detection

[This topic is pre-release documentation and is subject to change in future releases. Blank topics are included as placeholders.]

As a first line of defense, Forefront TMG provides mechanisms that inspect all traffic to detect individual packets or a specific minimal number of packets that were specially crafted for launching specific known types of attacks. These mechanisms are designed to detect the following:

• Spoofed source IP addresses

• Common intrusion attacks

• DNS attacks

• IP options

• IP fragments

• Invalid DHCP offers

Spoof detection

Forefront TMG inspects every packet received by a network adapter installed on the Forefront TMG server to determine whether its source IP address is valid for the network adapter at which it arrived. An IP address is considered valid for a specific network adapter if both the following conditions are true:

• The IP address is included in the IP address ranges assigned to the network of the network adapter through which it was received.

• The routing table indicates that traffic destined to that address can be routed through a network adapter belonging to that network.

If the source IP address of a packet is not considered valid, Forefront TMG drops the packet and generates an event that can trigger an IP Spoofing alert.

Intrusion detection

When Forefront TMG intrusion detection is enabled and offending packets are detected, they are dropped, and an event that triggers an Intrusion Detected alert is generated. By default, the Intrusion Detected alert is reset automatically after one minute, during which Forefront TMG continues to block offending packets but does not issue an alert. You can configure this alert to send you an e-mail notification when it is triggered. You can also enable logging of the dropped packets.

This protection mechanism can detect the following types of common attacks:

• Windows out-of-band (WinNuke) attack. The attacker launches an out-of-band denial-of-service (DoS) attack against a host protected by Forefront TMG. If the attack is successful, it causes the computer to fail or causes a loss of network connectivity on vulnerable computers.

• Land attack. The attacker sends a TCP SYN packet with a spoofed source IP address that matches the IP address of the targeted computer and with a port number that is allowed by the Forefront TMG policy rules so that the targeted computer will try to establish a TCP session with itself. If the attack is successfully mounted, it can cause some TCP implementations to go into a loop that causes the computer to fail.

• Ping-of-death attack. The attacker attaches a large amount of information that exceeds the maximum IP packet size to an Internet Control Message Protocol (ICMP) echo (ping) request. If the attack is successful, a kernel buffer overflows, causing the computer to fail.

• IP half-scan attack. The attacker repeatedly attempts to connect to a targeted computer but does not send ACK packets in response to SYN/ACK packets. During a normal TCP connection, the source initiates the connection by sending a SYN packet to a port on the destination system. If a service is listening on that port, the service responds with a SYN/ACK packet. The client initiating the connection then responds with an ACK packet, and the connection is established. If the destination host is not waiting for a connection on the specified port, it responds with an RST packet. Most system logs do not log completed connections until the final ACK packet is received from the source. Sending other types of packets that do not follow this sequence can elicit useful responses from the target host, without causing a connection to be logged.

• UDP bomb attack. The attacker attempts to send a UDP datagram with illegal values in certain fields, which causes some older operating systems to fail when the datagram is received. By default, no alert is configured for this type of attack.

• All port scan attack. The attacker attempts to count the services running on a computer by probing each port for a response. You can specify the number of ports that can be scanned before an event will be generated.

• Well-known port scan attack. The attacker attempts to count the services running on a computer by probing each port for a response. You can specify the number of ports in the range from 1 through 2048 that can be scanned before an event will be generated.

In this list, the name of each type of attack corresponds to an additional condition in the definition of the Intrusion Detected event. For each additional condition (type of attack), you can define and enable an alert, which specifies the actions to be taken in response to the event and is issued by the Microsoft Firewall service when all the conditions specified in the alert are met. The actions that can be triggered by an alert include sending an e-mail message, invoking a command, writing to a log, and starting or stopping Forefront TMG services.

RST attacks

Forefront TMG validates the sequence number in RST and SYN packets and drops packets that are out of sequence. This limits the ability of an attacker to terminate existing connections from other clients.

RST packets can be used to scan ports. When an attacker sends out an RST packet to a specific IP address and port, the attacker may receive no response at all or an ICMP Host Unreachable message. The absence of a response tells the attacker that the target host is probably running, while an ICMP Host Unreachable packet tells the attacker that the IP address probed is not assigned to a reachable host.

Detection of DNS attacks

The DNS Filter, which is installed with Forefront TMG, intercepts and analyzes all inbound DNS traffic destined for the Internal network and other protected networks. If DNS attack detection is enabled, you can specify that the DNS Filter will check for the following types of suspicious activity.

• DNS host name overflow. A DNS response for a host name exceeds a certain fixed limit (255 bytes). Applications that do not check the length of the host names may overflow internal buffers when copying this host name, allowing a remote attacker to execute arbitrary commands on a targeted computer.

• DNS length overflow. A DNS response for an IP address exceeds the specified length of 4 bytes. By crafting a DNS response with a longer value, some applications executing DNS lookups will overflow internal buffers, allowing a remote attacker to execute arbitrary commands on a targeted computer. Forefront TMG also checks that the value of RDLength does not exceed the size of the rest of the DNS response.

• DNS zone transfer. A client system uses a DNS client application to transfer zones from an internal DNS server.

When offending packets are detected, they are dropped, and an event that triggers a DNS Intrusion alert is generated. You can configure alerts that will be triggered for these events to notify you that an attack was detected. When the DNS Intrusion event is generated five times during one second for DNS zone transfer, a DNS Zone Transfer Intrusion alert is triggered. By default, after the applicable predefined alerts are triggered, they are not triggered again until they are reset manually.

Detection of POP buffer overflows

The POP Intrusion Detection Filter intercepts and analyzes POP traffic destined for the Internal network and other protected networks. Specifically, this application filter checks for POP buffer overflow attacks.

A POP buffer overflow attack occurs when a remote attacker attempts to gain root access of a POP server by overflowing an internal buffer on the server.

When offending POP traffic is detected, it is blocked, and an event that can trigger a POP Intrusion alert is generated. After this alert is triggered, Forefront TMG continues to reject offending POP traffic but does not trigger a new alert until the alert is reset manually.

IP option filtering

When IP option filtering is enabled, you can configure Forefront TMG to drop all IP packets with any IP option in their header, to drop all IP packets whose header contains any IP option that is in the list of selected IP options, or to drop all IP packets whose header contains any IP option that is not in the list of selected IP options.

The most problematic IP options are the source route options. TCP/IP supports source routing, which permits the sender of a packet to specify a series of IP addresses of gateways (routers) through which the packet must pass on its way to its destination. Each gateway (router) along the route uses this routing information to forward the packet to the next address on the way to its destination.

There are two source route IP options:

• Loose Source Route. The sender can specify a route in the form of a series of IP addresses through which the packet must pass on its way to its destination, but any number of intermediate gateways (routers) can be used to reach the next address specified in the route.

• Strict Source Route. The sender can specify an exact route in the form of a series of IP addresses through which the packet must pass on its way to its destination, but each gateway (router) must send the packet directly to the next address specified in the source route only through the directly connected network (rarely used).

The source route IP option in the IP header allows the sender to override routing decisions that are normally made by the routers between the source and destination computers. You can use source routing to map the network or to troubleshoot routing and communications problems. Source routing can also be used to force traffic through a route that provides the best performance.

Unfortunately, attackers can exploit source routing. For example, an intruder can use source routing to reach addresses on the Internal network that normally are not reachable from another network, by routing the traffic through another computer that is reachable from both the other network and the Internal network. This essentially causes a flood. You can improve Forefront TMG performance during a flood by disabling IP options filtering.

IP fragment filtering

A single IP packet can be divided into multiple packets of a smaller size, known as IP fragments. Forefront TMG can filter these fragments.

When IP fragment filtering is enabled, Forefront TMG drops all IP fragments. By default, the blocking of IP fragments is disabled.

The teardrop attack and its variants involve sending fragmented packets that will be reassembled in such a way that they may cause harm to the system. The teardrop attack works a little differently from the ping-of-death attack, but with similar results.

The teardrop program creates IP fragments, which are pieces of an IP packet into which an original packet can be divided as it travels through the Internet. The problem is that the offset and total length fields in these fragments, which are supposed to indicate the byte range of the original packet that is contained in each fragment, define overlapping ranges.

For example, normally the byte ranges indicated by the offset and total length fields in two fragments might appear as:

• Fragment 1: 100 (offset) - 300 (offset + total length)

• Fragment 2: 301 (offset) - 600 (offset + total length)

This indicates that the first fragment contains bytes 100 through 300 of the original packet, and the second fragment contains bytes 301 through 600.

Overlapping offset fields would appear something like this:

• Fragment 1: 100 (offset) - 300 (offset + total length)

• Fragment 2: 200 (offset) - 400 (offset + total length)

When the destination computer tries to reassemble these packets, it is unable to do so. It may fail, stop responding, or restart.

Kernel mode forwarding (IP routing)

When you configure IP options and IP fragment filtering, you can also configure kernel mode forwarding. Kernel mode forwarding improves performance. By default, kernel mode forwarding is enabled.

When kernel mode forwarding is disabled, Forefront TMG sends only the data (and not the original network packet) to the destination. Also, when disabled, Forefront TMG copies each packet and then resends it through the driver in user mode.

When kernel mode forwarding is disabled, Forefront TMG creates two additional sockets for each connection, resulting in increased resource consumption on the Forefront TMG server, increasing exposure to flood attacks. For this reason, if you disable kernel mode forwarding, we recommend that you deploy a router to protect Forefront TMG from TCP connection flood attacks.

When kernel mode forwarding is enabled, Forefront TMG acts as a router. Some filtering is performed by the driver in user mode on the traffic passing through Forefront TMG.

When kernel mode forwarding is enabled, Forefront TMG creates separate connections between the client and server. Forefront TMG fully parses and then reconstructs the IP and TCP headers, transferring only the data parts. If a malicious client attempts to exploit an IP or TCP vulnerability, Forefront TMG blocks the traffic, and the traffic does not reach the destination computer, which is protected by Forefront TMG.

Detection of DHCP poisoning

Forefront TMG can detect invalid DHCP offers. A DHCP offer is considered valid only if it is contained within a range of IP addresses associated with the network adapter on which the IP address is to be assigned. When an invalid DHCP offer is detected, Forefront TMG blocks assignment of the IP address to the network adapter and triggers an Invalid DHCP Offer alert. After this alert is triggered, Forefront TMG continues to reject invalid DHCP offers but does not trigger a new alert until the alert is manually reset.

If the network adapter receives the offered IP address, you can renew the lease at any time before it expires. When you do this, the enforcement mechanism is temporarily disabled, and a new ipconfig /renew command is issued. During this period, the DHCP anti-poisoning mechanism is disabled, and no offered addresses are dropped by Forefront TMG. After the adapters receive their addresses, Forefront TMG reactivates the mechanism.

DHCP offers may be dropped in the following scenarios:

• If you switch between two DHCP network adapters. For example, you switch between the network adapter that is connected to the Internal network and the network adapter that is connected to the External network.

• A DHCP network adapter was moved to a different network. For example, the Forefront TMG external network adapter was connected to a home network, behind a router connected to the Internet. When you replace the router with the Forefront TMG external network adapter, you must renew the DHCP address to allow the DHCP assignment.

After the assignment is allowed, you do not have to allow it again.

In some cases, you may want to move a network adapter from one network to another, using an address received from the DHCP server. You will want to accept an offer that Forefront TMG generally considers invalid.

Summary of alerts and events

The following table lists all the possible alerts that might be issued when an attack described in this topic is detected.

The following table lists events that are related to the attacks described in this topic. These events are displayed in Windows Event Viewer.

Copyright © 2009 by Microsoft Corporation. All rights reserved.