ISA Server Network Protection: Protecting Against Floods and Attacks

ISA Server Network Protection: Prot...

Collapse All

Microsoft Internet Security and Acceleration Server 2006

Businesses need to eliminate the damaging effects of malicious software (also called malware) and attackers by using comprehensive tools for scanning and blocking harmful content, files, and Web sites. Microsoft® Internet Security and Acceleration (ISA) Server 2006 Standard Edition and ISA Server 2006 Enterprise Edition help provide access protection with intrusion detection, flood mitigation, spoof detection, and other sophisticated attack detection features.

ISA Server 2006 Network Protection Feature Summary

The following table summarizes how ISA Server 2006 addresses the key concerns of information technology (IT) administrators tasked with safeguarding their IT environment.

Concerns	ISA Server 2006 features
Worms that propagate from user to user and network to network hurt users, partners, and customers.	Simplified client IP alert pooling and connection quotas improve resiliency against worms and minimize the impact that infected computers have on the network. Enhanced flood protection and intrusion detection features help block flood attacks, such as denial of service (DoS) and distributed denial of service (DDoS) attacks.
An increasing number of attacks on externally facing resources need to be combated.	Attack protection can be configured easily, to protect the corporate network from a wide variety of attacks, including Dynamic Host Configuration Protocol (DHCP) poisoning, intrusion detection, and IP fragmenting.
Protection is needed against spurious IP spoofing attacks.	IP spoofing protection is provided through a sophisticated anti-spoofing mechanism. ISA Server protects against IP spoofing by checking whether the packet's source IP address is valid.
Attacks go unnoticed for hours, or even days, underscoring the need for better ways to detect attacks when they occur, and take appropriate action.	Enhanced attack protection through comprehensive alert triggers and responses quickly notify administrators of network problems.

Scenario: How ISA Server Mitigates Attacks

Because of the prevalence of malicious hackers, the ability of ISA Server to combat attacks is key to your ability to protect your network. The default attack mitigation functionality helps you protect your network by blocking attempted attacks and alerting you to suspicious behaviors.

This combined detection and mitigation can help protect your network against a variety of potential attacks, some of which are listed in the following table.

Attack	Description
Internal worm propagation over TCP	Infected client computers attempt to connect to arbitrary addresses at a specific port to infect hosts with a known vulnerability.
Connection table exploit	An attacker uses many IP addresses or zombie hosts to create an excessive number of connections, which exhaust ISA Server resources to the point that ISA Server cannot be administered.
Pending Domain Name System (DNS) exploit during worm propagation	Infected clients attempt to connect to arbitrary addresses at specific ports. Because the ISA Server policy may be based on DNS names, ISA Server needs to reverse-resolve the arbitrary addresses.
Sequential TCP connections during flood attack	An attacker uses an internal host to launch a denial of service attack against ISA Server or another server behind ISA Server by sequentially opening and immediately closing many TCP connections, in an attempt to bypass the quota mechanism. This consumes a large amount of resources.
Hypertext Transfer Protocol (HTTP) DDoS using existing connections	An attacker sends HTTP requests at a high rate over a persistent (keep-alive) TCP connection. The ISA Server Web proxy needs to authorize every request. This consumes a large amount of resources from ISA Server. ISA Server includes this mitigation specifically for HTTP sessions, which are kept alive for a set period, with numerous connections as part of a single session.

Consider a scenario in which several computers on the corporate network become infected with a worm. These computers propagate the worm across the network. Each infected host produces a high rate of TCP connect requests to a specific port and to random IP addresses, trying to find other vulnerable computers to infect.

Meanwhile, ISA Server measures the allowed connect rate for each source IP address, raising alerts about a specific IP address, each belonging to an infected host. This alert is generated because the infected host exceeded a preconfigured threshold of allowed and denied connections per minute.

Before blocking this suspect IP address, ISA Server validates that the source IP address is not spoofed. If the source IP address is found to be malicious, ISA Server triggers an alert with information about the attack and about the attacker. From this point, ISA Server limits traffic from the offending host for one minute. After one minute, ISA Server again allows traffic from that IP address. If the threshold is again exceeded, and if you manually reset the alert, an alert is again triggered and traffic is blocked.

Only connection attempts that are allowed by the firewall policy are counted when triggering this alert. If a connection attempt is denied by the firewall policy, ISA Server counts the failed connection separately, and triggers a different alert.

A built-in ISA Server logging mechanism limits the system resources consumed by logging worm traffic, by raising an alert when a threshold is exceeded by a specific source IP address. This same mechanism also limits the total records logged per second, for traffic that is blocked by ISA Server policy. ISA Server logs the denied requests, until there are no longer enough resources. At this point, ISA Server stops logging denied packets. In addition, ISA Server may be working to trigger alerts. For this reason, ISA Server limits itself to a specific number of alerts per second.

If the IP address belongs to a user who is not intentionally launching a malicious attack, the user might call the help desk, complaining of loss of connectivity to the Internet. The help desk engineer reviews the ISA Server alerts, and notes that the user’s host violated the flooding policy. When the computer is checked, a worm is found on the computer. After the worm is removed from the host computer, the host no longer floods ISA Server with requests. Traffic from the host is no longer limited, and the client can access the Internet.

ISA Server Network Protection Features

ISA Server can help you mitigate the virus outbreaks and subsequent flooding of connections that are a prevalent corporate reality.

You can configure attack mitigation features, as described in the Configuring Attack Mitigation Features section. In addition, ISA Server includes built-in features to protect you against malicious attacks. These are described in the ISA Server Proconfigured Attack Protection section.

ISA Server triggers alerts, based on your configuration, which you can use to track and mitigate attacks. These alerts are detailed in the Alerting section.

Configuring Attack Mitigation Features

ISA Server includes attack mitigation features, which you can configure and monitor to help ensure that your network stays protected from malicious attacks. Depending on your specific deployment, you can configure the following features:

Flood attack and worm propagation mitigation features
HTTP connection limits
Protection against specific attacks, including IP packet protection, DHCP poisoning, and intrusion detection

This section describes the attack mitigation features.

Flood Attack and Worm Propagation Mitigation

A flood occurs when a malicious user attempts to attack a network in a variety of evolving methods. A flood attack may cause any of the following reactions:

Heavy disk load and resource consumption on the firewall
High CPU load
High memory consumption
High network bandwidth consumption

By configuring mitigation settings against flood attacks and worm propagation, you can limit the ability of malicious attackers to infiltrate your corporate network.

ISA Server limits the number of connections at any particular time. You can configure the limit, specifying a maximum number of concurrent connections. When the maximum number of connections is reached, any new client requests are denied.

The default configuration settings of flood mitigation help ensure that ISA Server can continue to function, even when under flood attack. This is accomplished because ISA Server classifies the traffic, and provides different levels of service to different types of traffic. Traffic that is considered malicious (with an intent to cause a flood attack) can be denied, while ISA Server continues to serve all other traffic.

The following table lists potential flood attacks and worm propagations, and briefly describes how ISA Server provides protection.

Attack	ISA Server mitigation	Defaults
Flood attack. A specific IP address attempts too many connections to many different IP addresses, causing a flood of connection attempts and terminations.	TCP connect requests per minute, per IP address. ISA Server mitigates flood attacks that occur when an attacking IP address sends numerous TCP connect requests. ISA Server also protects against worm propagations that occur when an infected host scans the network for vulnerable hosts.	By default, ISA Server limits the number of TCP requests per client to 600 per minute. You can configure custom limit exceptions for specific IP addresses. By default, this limit is set to 6,000 requests per minute.
Flood attack. A specific IP address tries to flood ISA Server by maintaining numerous TCP connections concurrently.	TCP concurrent connections per IP address. ISA Server mitigates a TCP flood attack that occurs when an offending host maintains numerous TCP connections with ISA Server or other servers.	By default, ISA Server limits the number of TCP concurrent connections per client to 160. You can configure custom limit exceptions for specific IP addresses. By default, this limit is set to 400 concurrent connections per client.
SYN attack. There is an attempt to flood ISA Server with numerous half-open TCP connections.	TCP half-open connections. ISA Server mitigates SYN attacks. In a SYN attack, an offending host sends TCP SYN messages without completing the TCP handshake.	By default, ISA Server limits the number of concurrent half-open TCP connections to half the number of concurrent connections configured for concurrent TCP connections. You cannot change this default.
DoS attack (HTTP). A specific IP address tries to launch a denial of service attack by sending numerous HTTP requests.	HTTP requests per minute, per IP address. ISA Server mitigates DoS attacks. In a DoS attack, an offending host sends numerous HTTP requests on the same TCP connection to victim Web sites.	By default, ISA Server limits the number of HTTP requests per client to 600 per minute. You can configure custom limit exceptions for specific IP addresses. By default, this limit is set to 6,000 requests per minute.
DoS attack (non-TCP). A zombie host tries to launch a denial of service attack by sending numerous non-TCP requests, which are denied by an ISA Server rule.	Non-TCP new sessions per minute, per rule. ISA Server mitigates non-TCP DoS attacks. In a non-TCP DoS attack, malicious hosts send numerous non-TCP packets to a victim server. The specific non-TCP traffic is allowed by an ISA Server rule.	By default, ISA Server limits the number of non-TCP sessions per minute to 1,000, for the specific protocol (rule).
User Datagram Protocol (UDP) flood attack. A specific IP address tries to launch a denial of service attack by opening numerous, concurrent UDP sessions.	UDP concurrent sessions per IP address. ISA Server mitigates UDP flood attacks. In a UDP flood attack, an offending host sends numerous UDP messages to victim hosts. When a UDP flood attack occurs, ISA Server discards older sessions, so that no more than the specified number of connections is allowed concurrently.	By default, ISA Server limits the number of concurrent UDP sessions per IP address to 160. You can configure custom limit exceptions for specific IP addresses. By default, this limit is set to 400 sessions per client.

For each flood mitigation limit that you configure, ISA Server monitors when the specified limit is passed. When this threshold is exceeded, ISA Server does the following:

Rejects new connect requests. After one minute, ISA Server resets the quota for this IP address. Traffic is no longer blocked. If the client again exceeds the quota, ISA Server once again blocks the traffic.

Note:
For TCP connections, no new connections are allowed after the flood mitigation limit is exceeded. For other connections (raw IP and UDP), older connections are terminated when the flood mitigation limit is exceeded so that new connections can be created.

Continues to serve traffic to existing connections.
Continues to serve system connections from the local host.

Before ISA Server blocks a specific IP address, it validates that the IP address is not spoofed.

When analyzing flood attacks, the most critical information is the IP address of clients generating suspicious traffic patterns. ISA Server can identify these IP addresses and notify you using alerts. To prevent overloading of information, ISA Server generates a single alert per offending IP address, even if that IP address continuously generates a suspicious traffic pattern (which is the typical traffic pattern for infected clients). An alert is generated no more than once per minute per IP address, to indicate that a limit has been exceeded (or that the limit is no longer being exceeded).

In ISA Server Enterprise Edition, custom limits that you configure for flood mitigations apply to all array members. When counting connections, the count is incremented against the side of the connection that initiated the connection.

DoS Attack on ISA Server Resources

In some DoS attacks, the malicious host attempts to attack the ISA Server firewall by exploiting its system resources. ISA Server mitigates this attack by lowering the idle connection time-out, when it is low on non-paged pool memory. If the attack continues, ISA Server blocks incoming new connections, when it is extremely low on non-paged pool memory. ISA Server also disconnects sessions that have been idle for at least six minutes.

Logging Flood Manipulation

ISA Server allows global configuration of the flood mitigation features. For all flood mitigations, you can configure logging of traffic blocked by flood mitigation.

The following table shows the error code returned by the Microsoft Firewall service that may appear in the Firewall log, when you enable logging for blocked traffic.

Result code	Hex ID	Details
WSA_RWS_QUOTA	0x80074E23	A connection was refused because a quota was exceeded.
FWX_E_RULE_QUOTA_EXCEEDED_DROPPED	0xC0040033	A connection was rejected because the maximum number of connections created per second for this rule was exceeded.
FWX_E_TCP_RATE_QUOTA_EXCEEDED_DROPPED	0xC0040037	A connection was rejected because the maximum connections rate for a single client host was exceeded.
FWX_E_DNS_QUOTA_EXCEEDED	0xC0040035	A DNS query could not be performed because the query limit was reached.

To enable logging

In the console tree of ISA Server Management, click General.
In the details pane, click Configure Flood Mitigation Settings.
On the Flood Mitigation tab, select Log traffic blocked by flood mitigation settings.

Exception List

Some flood mitigation quotas have two values:

One value applies to an exception list of IP addresses.
One value applies to all others.

For these flood mitigations, you can specify IP addresses to which the flood mitigation limits should not be applied. A custom limit will apply to these IP addresses.

You might configure an exception list for published servers, for array maintenance, and in some back-to-back firewall scenarios. For example, an exception list might apply to the Remote Management Computers computer set and array members (for ISA Server Enterprise Edition). The exception list might also include IP addresses of upstream or downstream proxy servers or other network address translation (NAT) devices (routers). These IP addresses may require many connections, and therefore need an increased limit.

You can configure quotas for IP address exception lists for the following flood mitigations:

TCP connect requests per minute, per IP address
TCP concurrent connections per IP address
HTTP requests per minute, per IP address
UDP concurrent sessions per IP address

Important:
An attacker may generate a flood attack by using spoofed IP addresses, which are included in the exception list. To mitigate this threat, we recommend that you deploy an Internet Protocol security (IPsec) policy between ISA Server and any trusted IP address included in the IP address exception list. An IPsec policy will enforce that traffic from these IP addresses is authenticated, thereby effectively blocking spoofed traffic.

Important:

An attacker may generate a flood attack by using spoofed IP addresses, which are included in the exception list. To mitigate this threat, we recommend that you deploy an Internet Protocol security (IPsec) policy between ISA Server and any trusted IP address included in the IP address exception list. An IPsec policy will enforce that traffic from these IP addresses is authenticated, thereby effectively blocking spoofed traffic.

To configure IP exceptions

In the console tree of ISA Server Management, click General.
In the details pane, click Configure Flood Mitigation Settings.
On the IP Exceptions tab, click Add.
In the Computer sets dialog box, add the desired computer sets.

Per IP Address Denied Log Entries and Alerts

The per IP address alert mechanism raises an alert if the number of denied packets from a specific IP address exceeds a preconfigured threshold. You can configure a general limit to apply to all IP addresses. You can also configure custom limit exceptions for a list of specific IP addresses.

When you configure this feature, alerts are triggered for all flood mitigations.

To configure alerts for blocked traffic

In the console tree of ISA Server Management, click General.
In the details pane, click Configure Flood Mitigation Settings.
On the Flood Mitigation tab, next to Set event trigger for denied packets, click Configure.
In Limit, type the limit for the flood mitigation.

Configuring HTTP Connection Limits

In addition to flood attack and worm propagation mitigation, you can also limit the number of Web proxy connections allowed simultaneously to the ISA Server computer. You can prevent attacks that overwhelm the system's resources. This is particularly useful when publishing Web servers. You can limit the number of computers that connect, while allowing for specific clients to continue connecting, even when the limit is surpassed.

You can configure Web proxy connection limits in ISA Server Management, for both published servers (on the Web listener) and for outgoing Web requests (on a specific network).

You can specify connection limits for a specific Web listener. Web listeners are used in Web publishing rules, and one Web listener may have multiple rules. When you specify a connection limit on the Web listener, you are limiting the number of connections allowed to Web sites published using the specific listener.

You can specify a connection limit on the Web proxy properties of a specific network object. Web Proxy Filter handles outgoing HTTP traffic on port 80. When you specify this connection limit on a specific network, you limit the number of concurrent outgoing Web connections that are allowed on a specific network at any specific time.

Configuring Attack Protection

In addition to flood attack and worm propagation mitigation, and HTTP connection limits, ISA Server includes a variety of mechanisms that you can configure to help protect your network from attacks:

IP packet protection
Broadcast protection
DHCP poisoning protection
Intrusion detection
Spoof detection

The following sections detail these mechanisms.

Configuring IP Packet Protection

You can configure how ISA Server handles IP packets, by configuring the following:

IP fragment filtering
IP routing
IP options

Configuring IP Fragment Filtering

A single IP datagram can be divided into multiple datagrams of a smaller size, known as IP fragments. ISA Server can filter these fragments.

All fragmented packets are dropped when ISA Server filters packet fragments. The teardrop attack and its variants involve sending fragmented packets and then reassembling them in such a way that may cause harm to the system. The teardrop attack works a little differently from the ping of death attack, but with similar results.

The teardrop program creates IP fragments, which are pieces of an IP packet into which an original packet can be divided as it travels through the Internet. The problem is that the offset fields on these fragments, which are supposed to indicate the portion (in bytes) of the original packet that is contained in the fragment, overlap.

For example, normally offset fields in two fragments might appear as:

Fragment 1:  (offset) 100 - 300
Fragment 2:  (offset) 301 - 600

This indicates that the first fragment contains bytes 100 through 300 of the original packet, and the second fragment contains bytes 301 through 600.

Overlapping offset fields would appear something like this:

Fragment 1:  (offset) 100 - 300
Fragment 2:  (offset) 200 - 400

When the destination computer tries to reassemble these packets, it is unable to do so. It may fail, stop responding, or restart.

To block IP fragments

In the console tree of ISA Server Management, click General.
In the details pane, click Define IP Preferences.
On the IP Fragments tab, select Block IP fragments.

Note:
Fragment filtering can interfere with streaming audio and video. In addition, Layer Two Tunneling Protocol (L2TP) over IPsec connections may not be successfully established because packet fragmentation may take place during certificate exchange. Disable fragment filtering if you have problems with streaming media and IPsec-based virtual private network (VPN) connections.

Configuring IP Routing

To protect against unwarranted connection termination, ISA Server implements the following:

Connection flood mitigation. ISA Server validates that the three-way handshake packets required in a sequence are valid. This avoids establishing TCP connections to or through ISA Server from spoofed source IP addresses.
RST attack mitigation. ISA Server validates the sequence number on RST and SYN packets. This mitigates the ability of an attacker to terminate existing connections from other clients.

The underlying operating system also includes a similar mechanism. To enable the operating system mechanism, which is useful in server publishing or chaining scenarios, disable the ISA Server IP routing feature.

When IP routing is disabled, ISA Server sends only the data (and not the original network packet) to the destination. Also, when disabled, ISA Server copies each packet, and then resends it through the driver in user mode.

Important:
When IP routing is disabled, ISA Server creates two additional sockets for each connection, resulting in increased resource consumption on the ISA Server firewall, increasing ISA Server exposure to flood attacks. For this reason, if you disable IP routing, we recommend that you deploy a router to protect ISA Server from TCP connection flood attacks.

When IP routing is enabled, ISA Server acts as a router. Some filtering is performed by the driver in user mode on the traffic passing through ISA Server.

When IP routing is enabled, ISA Server creates separate connections between the client and server. ISA Server fully parses and then reconstructs the IP and TCP headers, transferring only the data parts. If a malicious client attempts to exploit an IP or TCP vulnerability, ISA Server blocks the traffic, and the traffic does not reach the destination computer, which is protected by ISA Server.

To disable IP routing

In the console tree of ISA Server Management, click General.
In the details pane, click Define IP Preferences.
On the IP Routing tab, clear the Enable IP routing check box.

Configuring IP Options

You can configure ISA Server to refuse all packets that have the flag IP Options in the header.

The most problematic options are the source routing options. TCP/IP supports source routing, which is a means to permit the sender of network data to route the packets through a specific point on the network. There are two types of source routing:

Strict source routing. The sender of the data can specify the exact route (rarely used).
Loose source record route. The sender can specify certain routers (hops) through which the packet must pass.

The source route option in the IP header allows the sender to override routing decisions that are normally made by the routers between the source and destination computers. You can use source routing to map the network or to troubleshoot routing and communications problems. Source routing can also be used to force traffic through a route providing the best performance.

Unfortunately, attackers can exploit source routing. For example, an intruder can use source routing to reach addresses on the Internal network that normally are not reachable from other networks, by routing the traffic through another computer that is reachable from both the other network and the Internal network. This essentially causes a flood. You can improve ISA Server performance during a flood by disabling IP options filtering.

To disable IP options filtering

In the console tree of ISA Server Management, click General.
In the details pane, click Define IP Preferences.
On the IP Options tab, clear the Enable IP options filtering check box.

Configuring DHCP Poisoning Protection

ISA Server can detect invalid DHCP offers. A DHCP offer is considered valid only if it is contained in the range of the network object associated with the network adapter on which the IP address was assigned. When an invalid DHCP offer is detected, ISA Server triggers an Invalid DHCP offer alert and ignores the invalid DHCP offer.

ISA Server continues to protect against DHCP offer attacks, even after you acknowledge this alert.

Note:
For ISA Server Enterprise Edition, if you accept a new IP address for the Internal network, verify that the Configuration Storage server can be accessed through this IP address.

ISA Server maintains a list of allowed IP addresses for each DHCP network adapter. The allowed addresses are from the address set of a network that contains the adapter's address. When ISA Server receives a DHCP offer packet, it checks that the offer is within the range of allowed addresses. If the validation fails, the packet is dropped, and an Invalid DHCP offer alert is triggered.

If the network adapter receives the offered address, you can renew DHCP addresses. When you do this, the enforcement mechanism is temporarily disabled, and a new ipconfig /renew command is issued. During this period, no offered addresses are dropped by ISA Server. After the adapters receive their addresses, ISA Server reactivates the mechanism.

DHCP offers may be dropped in the following scenarios:

If you switch between two DHCP adapters. For example, you switch between the adapter that is connected to the Internal network and the adapter that is connected to the External network.
A DHCP adapter was moved to a different network. For example, the ISA Server external network adapter was connected to a home network, behind a router connected to the Internet. When you replace the router with the ISA Server external network adapter, you must renew the DHCP address, to allow the DHCP assignment.

After the assignment is allowed, you do not have to allow it again.

In some cases, you may want to move a network adapter from one network to another, using an address received from the DHCP server. You will want to accept an offer that ISA Server generally considers invalid.

To accept a DHCP offer

At a command prompt, type ipconfig /renew. (You will receive an error message.)
In ISA Server Management, configure the invalid DHCP offer alert.
Verify that the IP address for the network adapter is appropriately assigned.
Check that the network object associated with the network adapter reflects the new IP address.

Configuring Intrusion Detection

The ISA Server intrusion detection mechanism identifies when an attack is attempted against your network and performs a set of configured actions, or alerts, in case of an attack. To detect unwanted intruders, ISA Server compares network traffic and log entries to well-known attack methods. Suspicious activities trigger alerts. Actions include connection termination, service termination, e-mail alerts, and logging.

The following table lists the alerts that ISA Server can trigger if intrusion detection is enabled.

Attack	Description
All ports scan attack	This alert notifies you that an attempt was made to access more than the preconfigured number of ports. You can specify a threshold, indicating the number of ports that can be accessed.
Enumerated port scan attack	This alert notifies you that an attempt was made to count the services running on a computer by probing each port for a response. If this alert occurs, you should identify the source of the port scan. Compare this with the services that are running on the target computer. Also, identify the source and intent of the scan. Check the access logs for indications of unauthorized access. If you detect indications of unauthorized access, you should consider the system compromised and take appropriate action.
IP half scan attack	This alert notifies you that repeated attempts to send TCP packets with invalid flags were made. During a normal TCP connection, the source initiates the connection by sending a SYN packet to a port on the destination system. If a service is listening on that port, the service responds with a SYN/ACK packet. The client initiating the connection then responds with an ACK packet, and the connection is established. If the destination host is not waiting for a connection on the specified port, it responds with an RST packet. Most system logs do not log completed connections until the final ACK packet is received from the source. Sending other types of packets that do not follow this sequence can elicit useful responses from the target host, without causing a connection to be logged. This is known as a TCP half scan, or a stealth scan, because it does not generate a log entry on the scanned host.
Land attack	This alert notifies you that a TCP SYN packet was sent with a spoofed source IP address and port number that matches that of the destination IP address and port. If the attack is successfully mounted, it can cause some TCP implementations to go into a loop that causes the computer to fail.
Ping of death attack	This alert notifies you that an IP fragment was received with more data than the maximum IP packet size. If the attack is successfully mounted, a kernel buffer overflows, which causes the computer to fail.
UDP bomb attack	This alert notifies you that there was an attempt to send an illegal UDP packet. A UDP packet that is constructed with illegal values in certain fields causes some older operating systems to fail, when the packet is received. If the target computer fails, it is often difficult to determine the cause.
Windows out-of-band attack	This alert notifies you that there was an out-of-band denial of service attack attempted against a computer protected by ISA Server. If mounted successfully, this attack causes the computer to fail or causes a loss of network connectivity on vulnerable computers.

ISA Server includes intrusion detection filters:

The DNS intrusion detection filter works with DNS server publishing rules. The filter intercepts and analyzes all inbound DNS traffic destined for the published network.
The POP intrusion detection filter checks for POP3 buffer overflow attacks.

DNS Intrusion Detection Filter

The DNS filter, installed with ISA Server, intercepts and analyzes DNS traffic destined for the published network. The DNS filter checks for DNS length overflow and optionally blocks zone transfers.

In addition, you can configure which of the following DNS attacks triggers alerts:

DNS host name overflow. A DNS host name overflow occurs when a DNS response for a host name exceeds a certain fixed length (255 bytes). Applications that do not check the length of the host names may return overflow internal buffers when copying this host name, allowing a remote attacker to execute arbitrary commands on a targeted computer.
DNS length overflow. DNS responses for IP addresses contain a length field, which should be 4 bytes. By formatting a DNS response with a larger value, some applications executing DNS lookups will overflow internal buffers, allowing a remote attacker to execute arbitrary commands on a targeted computer. ISA Server also checks that the value of RDLength does not exceed the size of the rest of the DNS response.
DNS zone transfer. A DNS zone transfer occurs when a client system uses a DNS client application to transfer zones from an internal DNS server.

POP Intrusion Detection Filter

The POP Intrusion Detection filter intercepts and analyzes POP traffic destined for the Internal network. Specifically, the application filter checks for POP buffer overflow attacks.

A POP buffer overflow attack occurs when a remote attacker attempts to gain root access of a POP server by overflowing an internal buffer on the server.

ISA Server Preconfigured Attack Protection

ISA Server includes preconfigured protection against specific attacks. This includes spoof detection and broadcast protection.

These features are described in the following sections.

Spoof Detection Alerts

Every time a network adapter receives a packet, ISA Server checks whether the packet's source is spoofed. ISA Server checks whether the packet's source IP address is a valid IP address for the specific network adapter that received it. If the address is not considered valid, ISA Server alerts that an IP spoofing attack has occurred.

An IP address is considered valid for a specific network adapter if both the following conditions are true:

The IP address resides in the network of the adapter through which it was received.
The routing table indicates that traffic destined to that address may be routed through one of the adapters belonging to that network.

Note:
For ISA Server Enterprise Edition, spoof detection is not applied to traffic from an intra-array address. All traffic from the intra-array addresses passes directly to the engine for a policy check.

Consider the following scenario, in which a network includes IP addresses in the range 10.X.X.X. The routing table shows the following.

Network  Netmask   DestinationGateway interface
10.0.0.0 255.0.0.0   10.0.0.1  10.1.1.1
20.0.0.0 255.0.0.0   20.0.0.1  10.1.1.1
0.0.0.0  0.0.0.0     140.0.0.1 140.1.1.1

Packets received on interface 10.1.1.1 with source IP addresses in the range from 10.0.0.1 through 10.255.255.255 will not be discarded as spoofed, because those addresses can be routed back through this interface, and they belong to the address range of the network. However, packets with source IP addresses that are outside this range (including the range from 20.0.0.1 through 20.255.255.255, which can be routed through the interface 10.1.1.1) will be dropped as spoofed, because they do not belong to the network.

ISA Server also makes sure that all packets that are sent through an adapter have a valid destination IP address. This prevents packets from being routed through the wrong adapters in case of a routing table configuration problem.

When ISA Server detects a spoofed packet, ISA Server triggers an alert indicating the reason that the packet is considered spoofed. You should carefully review the alert, and attempt to address the issue by doing one of the following:

Fixing potential configuration errors. Verify that packets from the specific IP address should be considered spoofed. If not, determine why ISA Server considers these packets spoofed.
Blocking traffic from the IP address. If traffic from the IP address should be considered spoofed, block all access from that IP address.

Broadcast Protection

Broadcasting is the sending of a single message (datagram) to multiple recipients, using a connectionless protocol such as UDP. There are three types of broadcast addresses:

Limited broadcast address. The limited broadcast address is 255.255.255.255.
Network broadcast address. The network broadcast address has an IP address of all one bits and a specific network ID. For example, consider a class A network 22.0.0.0. Its network broadcast address is 22.255.255.255.
Subnet broadcast address. The subnet broadcast address has an IP address of all one bits and a specific subnet ID. For example, consider a class A network 22.0.0.0 with a subnet mask 255.255.0.0. The address 22.0.255.255 is a subnet broadcast.

ISA Server does not allow any broadcast messages to be sent between network adapters on the ISA Server computer. ISA Server determines whether a rule applies to broadcast addresses:

If the destination address is not a subnet broadcast address, ISA Server considers the address as the subnet broadcast address of the network adapter on the ISA Server computer on which the packet was received.
If the packets are an incoming broadcast (to the Local Host network), ISA Server considers the destination as the network adapter on the ISA Server computer (Local Host).
If this is an outgoing broadcast (from the Local Host network), ISA Server considers the source as the network adapter on the ISA Server computer (Local Host).

For example, assume a service listens on UDP port 1500 of the Local Host network (the ISA Server computer) on the network adapter with subnet address 22.0.1.1. To allow broadcast traffic to that service from, for example, the Internal network, you must create an access rule that allows traffic on UDP port 1500 from the Internal network to the subnet broadcast address 22.0.255.255. Alternatively, you can create an access rule that allows UDP port 1500 traffic from the Internal network to the Local Host.

Alerting

ISA Server can trigger alerts and log suspicious behavior when detecting attacks. This section lists some attack detection alerts.

Flood Mitigation Alerts

The following table lists all the possible alerts that might be generated in case of a flood attack.

Alert title	Event description
Low Non-Paged Pool	The size of the free non-paged pool is below the system-defined minimum.
Low Non-Paged Pool Recovered	The size of the free non-paged pool no longer exceeds the system-defined minimum.
Pending DNS Requests Resource Usage Limit Exceeded	The percentage of threads used for pending DNS requests out of the total number of available threads exceeds the system-defined maximum.
Pending DNS Requests Resource Usage within Limits	The percentage of threads used for pending DNS requests out of the total number of available threads is below the system-defined maximum, and connections that require DNS name resolution can be accepted.
TCP Connections per Minute from One IP Address Limit Exceeded	The number of TCP connections per minute allowed from one IP address is exceeded.
Concurrent TCP Connections from One IP Address Limit Exceeded	The number of concurrent TCP connections allowed from one IP address is exceeded.
Non-TCP Sessions from One IP Address Limit Exceeded	The number of non-TCP sessions allowed from one IP address is exceeded.
Connection limit for a rule was exceeded	The number of non-TCP sessions per second allowed by one rule exceeds the configured limit.
Denied Connections per Minute from One IP Address Limit Exceeded	The number of connections per minute from one IP address blocked by the firewall policy exceeds the configured limit.
Global Denied Sessions per Minute Limit Exceeded	The total number of blocked TCP and non-TCP sessions per minute exceeds the configured limit.
HTTP Requests from One IP Address Limit Exceeded	The number of HTTP requests per minute from one IP address exceeds the configured limit.

The following table lists some events that ISA Server triggers when a flood mitigation limit is exceeded.

Event ID	Message
15112	The client clientname exceeded its connection limit. The new connection was rejected.
15113	ISA Server disconnected the following client: clientname because its connection limit was exhausted.
15114	ISA Server disconnected a connection because its connection limit was exceeded.
15116	The request was denied because the number of connections per second allowed for a rule was exceeded.
15117	The request was denied because the number of connections per second allowed for the rulename rule was exceeded.

The ISA Server events generated may trigger the ISA Server alert warnings listed in the following table.

Alert warning	Description
Connection limit exceeded	Connection limits are exceeded for an IP address.
Connection limit for a rule was exceeded	The number of connections per second for a rule is exceeded.
Pending DNS Requests Resource Usage Limit Exceeded	The percentage of threads used for pending DNS requests out of the total number of available threads exceeds the system-defined maximum.
Pending DNS Requests Resource Usage Limit within Limits	The percentage of threads used for pending DNS requests out of the total number of available threads is below the system-defined maximum, and connections that require DNS name resolution can be accepted.

Attack Protection Alerts

The following table lists alerts that might be generated in case of other attacks.

Alert title	Event description
DHCP Anti-Poisoning Intrusion Detection Disabled	The DHCP anti-poisoning intrusion detection mechanism is disabled.
DNS Intrusion	A host name overflow, length overflow, or zone transfer attack occurred.
DNS Zone Transfer Intrusion	A zone transfer attack occurred.
Intrusion Detected	An intrusion was attempted by an external user.
Invalid DHCP offer	The DHCP offer IP address is not valid.
IP Spoofing	The IP packet source address is not valid.
POP Intrusion	POP buffer overflow detected.
SYN Attack	ISA Server detected a SYN attack.

Best Practices

This section describes some best practice guidelines that you should follow to configure ISA Server to better protect your network.

Detecting Flood Attacks

Follow these guidelines to detect whether your network is experiencing a flood attack:

Check for a sudden surge in CPU use, increased memory consumption, or very high logging rates on the ISA Server computer. These symptoms often indicate that ISA Server is experiencing a flood attack.
Check for relevant alerts.
Use the ISA Server logs to inspect traffic, validating that the traffic is expected and allowed:
- Inspect traffic initiated by any type of client (Firewall client, Web Proxy client, SecureNAT client, VPN client, and external clients). Validate that the traffic is expected and allowed.
- Inspect traffic regardless of its direction (inbound or outbound).
Identify potentially compromised clients, based on the following criteria:
- Clients that produce a high volume of connections or requests per second
- Clients that extensively consume network bandwidth (bytes per second)
- Clients that produce connection failures to different destination addresses at a high rate
- Clients that attempt to bypass the ISA Server firewall policy

If you determine that the ISA Server computer is experiencing a flood attack, use the log viewer to determine the source of the offending traffic. Specifically, look for the following:

Log entries for denied traffic. Pay special attention to traffic that is denied because the quota is exceeded, packets are spoofed, and packets contain corrupted CHECKSUM. These usually are indicative of a malicious client. In ISA Server Standard Edition, connections that are terminated due to exceeding the connections limit will have a result code of 0x80074E23. In ISA Server Enterprise Edition, the result will appear as text, which clearly indicates the connection termination reason.
Logs that indicate numerous connections that are created and then immediately closed. This often indicates that a client computer is scanning an IP address range for a specific vulnerability.

We recommend that you limit the number of connections, because this can help prevent flood attacks. When a UDP or raw IP flood attack occurs, many requests are sent from spoofed source IP addresses, eventually resulting in a denial of service.

Mitigating Flood Attacks

When an alert occurs, determine whether your network is being attacked, or whether there is simply a heavy load of valid traffic. If the limit was exceeded due to malicious traffic, try the following:

If the malicious traffic appears to originate from the Internal network, this may indicate a virus on the Internal network. Identify the source IP address, and disconnect the computer from the network immediately.
If the malicious traffic appears to originate from a small range of IP addresses on an External network, create a rule denying access to a computer set that includes the source IP addresses.
If the traffic appears to originate from a large range of IP addresses, evaluate the overall status of your network. Consider setting a smaller connection limit, so that ISA Server can better protect your network.
If the limit has been exceeded due to heavy load, consider setting a higher connection limit.

Another way to mitigate attacks is to create a new network. The network should include the IP addresses of the infected clients, which you detected based on the logs and alerts. Then, exclude these IP addresses from networks to which they originally belonged (the Internal network). You essentially create a mismatch between the routing table on the ISA Server firewall and the Internal network. These IP addresses are considered spoofed, and traffic to and from the IP addresses is dropped accordingly.

Logging in Case of Attack

Each time a flood mitigation limit is exceeded, ISA Server generates an alert, indicating the IP address of the offending client. After you identify the list of offending IP addresses, to prevent unnecessary logging, perform the following procedure, to help improve the performance of ISA Server during the flood.

To improve ISA Server performance during a flood

Disable logging of flood-related log entries. Follow the instructions in the Logging Flood Mitigations section in this document.
Disable logging. Disable logging either on the specific rule that matches the flood or altogether until the flood attack is stopped.
Reconfigure the Connections Limit alerts (or any other types of alerts that may be triggered repeatedly as a result of the specific attack) to Manually Reset.

Important:
When you disable the log for denied log entries, you can identify only potential alerts. The tips listed in the Detecting Flood Attacks section in this document will not be relevant.

Customizing Flood Mitigation Limits

We recommend that you use the default flood mitigation limits. However, these defaults might not be appropriate in the following network address translation (NAT) scenarios:

Back-to-back perimeter scenario. In this scenario, the internal ISA Server firewall applies NAT to outgoing requests from internal clients, and requests are forwarded to the edge firewall with the address of the internal ISA Server firewall. To the edge server, all connections appear to be from a single client. For example, 20 requests from different clients appear to the edge ISA Server computer as 20 requests from the same IP address. The default connection limit for this IP address may be quickly exhausted.
Firewall or Web chaining scenario. Web chaining routes Web proxy requests to an upstream proxy server. Firewall chaining configures the downstream ISA Server computer as a SecureNAT client or Firewall client of the upstream proxy. In both cases, NAT is applied to client requests that are routed to an upstream server. The upstream server will consider different client requests from the same network as having the same IP address. Again, the default connection limit for this IP address may be quickly exhausted.
Site-to-site VPN scenario. Connection limits are enforced for site-to-site virtual private network (VPN) connections. Although NAT is applied to traffic between the remote networks, the IP address that replaces the internal addresses is automatically assigned a custom limit. An exceeded limit error does not generally occur in this scenario.

To respond to a flood mitigation alert

Determine whether your network is being attacked, or whether there is a heavy load of valid traffic. Use network monitoring tools to determine this.
If the limit is exceeded due to a heavy load of non-TCP traffic, consider setting a higher per-rule connection limit. If the limit is exceeded due to malicious traffic, try the following:
- If the malicious traffic appears to originate from the Internal network, this may indicate a virus on the Internal network. Identify the source IP address, and disconnect the computer from the network immediately.
- If the malicious traffic appears to originate from a small range of IP addresses on an Internal or External network, create a rule denying access to a computer set that includes the source IP addresses.
- If the traffic appears to originate from a large range of IP addresses, evaluate the overall status of your network. Consider setting a significantly smaller connection limit, so that ISA Server can better protect your network, while still providing services to clients who are not malicious.
If you publish more than one UDP-based or raw IP-based service to the External network, you should configure smaller limits, to help keep your network secure from flood attacks.