In this paper, Anton looks at network intrusion systems, IDS-triggered countermeasures, what are they, how they can be triggered and when they should not be triggered.

Intrusion detection systems (IDS) seem to be one of the fastest growing technologies within the security space. Together with firewalls and vulnerability scanners, intrusion detection forms one of the cornerstones of modern computer security. In the commonly mentioned prevention-detection-response philosophy, IDSs take an honorable place for [sometimes] effective detection of threats let through by prevention technologies such as firewalls.

However, there are attempts to position IDS products as the technology that can stop or prevent network attacks. It is easy to forget that 'D' stands for detection and not for deterrent or deflection. This article will investigate those attempts in the Linux world.

On the technical level, most network IDS are set to send alerts upon seeing a known pattern (signature-based IDS) or some traffic anomaly (anomaly-based IDS) indicating an attack. While some programs can look only at packet level data (such as attack string present in one packet), many others are TCP connection-aware, are able to look at TCP streams and also can reassemble fragmented IP packets.

In this paper we will look at IDS-triggered countermeasures, what are they, how they can be triggered and when they should not be triggered. Lets first assume we have an IDS that looks at traffic passing through the wire (see Picture 1). That corresponds to the majority of deployed IDS. What actions can thus deployed IDS invoke? First, it can send an alert that can be handled by outside programs to accomplish pretty much any action. Seconds, IDS itself can try to influence the traffic that passes by.


IDS Alert Actions

The first opportunity presents a rich spectrum of possible actions: IDS alerts can be send via pager, SMS message and other mechanisms to trigger humans into actions. On the other hand, the alerts can cause some software to activate, such as to block an attack by imposing a new firewall rule or even to launch a counter-attack or an investigative probe towards an attacker. The practice of applying firewall rules based on IDS alerts is sometimes known as "shunning".

IDS running inside the network (as on Picture 1) or on the firewall machine (listening to the internal network interface) is in a good position to block an attack. IDS has to notify the firewall (locally or over the network) to apply the rule blocking the source of detected malicious packet.

What are the requirements for such as setup? Some follow below:


  • IDS should transmit the request as fast as possible since the damage from attack could increase with time and do it over reliable channel unaffected by the above attack
  • IDS might want to block the address entirely or for a certain time and for certain protocols only
  • Most important at all, IDS should not block on false-positives or on signatures that can be faked (spoofed)

What do the above requirements mean for real-life IDS deployment. The evident conclusion, for example, is that UDP- or ICMP-based signatures should never be used for blocking since the packets can be spoofed. TCP packets should only cause a block if there is an established connection between an attacker machine and the protected machine. Otherwise, malicious hackers can cause significant damage to your network by using spoofed packets (such as by sending "attacks" seemingly originating from root DNS servers, upstream providers or partner's networks). See Understanding IDS Active Response Mechanisms for more details. Spoofing the TCP connection with a completed three-way handshake is much harder and is often appears impossible. Even with full TCP connection tracking, having a "whitelist" of addresses that should never be blocked is strongly suggested.

In addition, blocking on signatures that have a known high false-positive rate is not recommended due to the obvious risks of blocking non-malicious traffic.

Moreover, while it is too late to stop an attack detected by the IDS (since the packet has proceeded to its destination within the internal network), promptly blocking the subsequent communication is of crucial importance. It takes only few seconds for the modern automatic penetration tools to get access, elevate the privileges on the attacked system and deploy a backdoor or a rootkit. There are several options for sending commands over to the firewall.

For single box setup, the options are: IDS itself runs the command to implement the block (for example, using Linux ipchains/iptables) or some program reads IDS log files and then blocks the connection. Both methods have some problems that call for a careful design before implementation. The former method should not make IDS wait for the response from the firewall (unless IDS is a multi-threaded process), since it will slow down the detection. On the other hand, the latter method might introduce a delay since IDS should create an alert and write it to a log file. Then blocking program checks the IDS log file (presuming the data was already flushed to disk) and implements the block. Both methods can be used if their limitations are understood.

For the IDS and firewall on different machines, the situation is similar. While makeshift solutions are possible the delay issue should be handled. For example, using insecure out-of-band communication (such as direct ethernet cable from firewall to IDS) is better than running a command via SSH over the monitored network (secure but slow). Establishing SSH session takes too much time. It is reported that Checkpoint OPSEC interface is handling these issues very well, since it was designed for that purpose.

The issue of reliable transmission calls for out-of-band management. The problem does not manifest itself for IDS and firewall on the same machine, but for separate systems the separate subnet should be used for their communication (as shown on Picture 1). Using the protected network for the communication is highly discouragd, since attackers migth have means to disrupt it.

Time-limit on shuns is essential as well. For example, if attacker is a modem dial-up user his or her IP address might be assigned to non-malicious users after modem session terminates.

In any case, an ability to disable automatic blocking should be provided.


Direct IDS Intervention

The second reaction option mentioned above is direct interaction by IDS itself. IDS can send a TCP reset (to one or both communicating parties) or an ICMP message to attack source (ICMP_NET_UNREACH, ICMP_HOST_UNREACH or ICMP_PORT_UNREACH types can be used). TCP reset is a TCP packet with RST flag set, that is usually used in the teardown phase of the network connection.

If signatures that trigger a response were carefully selected to have low false-positives, the white list (as for firewall blocking) is not really essential, since only a single connection will be terminated.

It should be noted, that if IDS sends TCP RSTs or ICMPs (or anything else, for that matter), it provides an avenue for possibly identifying the IDS type and operating system. Unless TCP/IP options on the packet can be played with, the IDS presence and type will be uncovered by an attacker.

Implementing Countermeasures

Now lets consider how open-source IDS Snort implements the above options. It is worth noting that most commercial IDS (such as Cisco Secure IDS, Cisco PIX IDS and ISS products) also implement blocking based on alarms.

Several programs that read snort log files and generate firewall rules based on Snort alerts exist. Guardian appears to be the most flexible from open-source tools. It implements time-based blocking, supports several firewalls (Linux iptables/ipchains, FreeBSD ipfw, ipfilter, Checkpoint FW-1, Cisco PIX firewall), features a "white list" and can even do blocking on a machine with no packet-filtering software using black-hole routing. Guardian runs as a daemon alongside snort and reads alerts from standard snort file (/var/log/snort/alert in default snort configuration). It will block on ALL logged events and port scans, thus Snort's signature set should be severely cleaned up. In addition, snort must be run with "-z est" flags to only alert on attacks present in established TCP connections. Otherwise, spoofed attacks might turn this setup into a "DoS machine". Even with "-z est", white list (/etc/guardian.ignore) must be populated with hosts for which connectivity is essential (example of defense in-depth).

Snort have an ability to send output via custom output plug-ins. One such plug-in called snortsam supports Checkpoint Firewall blocking. Encrypted communication, time-based blocking and other features are supported. Unlike the previous program, Snortsam uses a rule target to decide whether to block based on alert. Thus it is possible to run with a full signature set and only block on selected alerts. The targeting is extremely flexible. See for full details.

Snort can send TCP RSTs and various ICMPs to terminate the connection. The code makes use of "resp" keyword added to a rule. Thus, snort can respond only to selected attack signatures. The excerpt below was used for testing:

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS 80 (msg:"WEB-IIS cmd.exe 
access"; flags: A+; content:"cmd.exe"; resp: icmp_all,rst_all; nocase; 
classtype:web-application-attack; sid:1002; rev:2;)

In the above example, all possible response messages are sent. Namely, both sender and receiver of an attack packet get a TCP reset and a sender gets 3 ICMPs (port, host, network unreachable). In fact, ICMPs can be used to stop UDP transmissions and TCP RSTs are used to tear down TCP sessions. That is how it looks in packet capture (tcpdump and browser run on host "anton", snort runs on host "fw"):


In our tests, snort (v 1.8.4 and beta v. 1.9.1) does not always kill the HTTP connection using the RST and/or ICMPs. In most of the cases connection is reset and sometimes it remains running and the file (dummy " cmd.exe" placed on Apache web server) is successfully downloaded. The possible explanation is that RST arrives too late for the connection to be reset since the response from server comes earlier with the right sequence number. The delayed RST is then discarded. Thus RST/ICMP is not a reliable security mechanism (exactly as claimed in the snort documentation).



To conclude, IDS ability to respond to attacks is a useful feature. With expert tuning, it can provide added security to the IT environment. Inherent dangers and weaknesses of auto-response should then also be addressed. The best compromise is to only use IDS active response on high-risk alerts with low false positives (such as backdoor responding, known virus transmission, DoS master-slave communication, etc).

Interesting development in the active IDS field might come from "inline IDS" systems (shown on Picture 3). In this setup, IDS actually IS able to drop the offensive packet, and not only the connection that ensues sice IDS engine is actually making the routing decision. Snort-based Hogwash is an example of such system.


About the Author

Anton Chuvakin, Ph.D. is a Senior Security Analyst with netForensics (, a security information management company that provides real-time network security monitoring solutions.