Intrusion Detection is the process and methodology of inspecting data for malicious, inaccurate or anomalous activity. At the most basic levels there are two forms of Intrusion Detection Systems that you will encounter: Host and Network based. In today's hostile Internet world, network administrators need to have a watchful eye over the networks they administer. Intrusion Detection Systems can assist the administrator with notification when malicious or suspicious activity occurs. This paper will introduce Intrusion Detection Systems as a layer in a network's security infrastructure.

Intrusion Detection is the process and methodology of inspecting data for malicious, inaccurate or anomalous activity. At the most basic levels there are two forms of Intrusion Detection Systems that you will encounter: Host and Network based. Host based can use logs as its data source, while Network based will use network traffic as its data source.

Host Based:

Host based Intrusion Detection Systems role is to identify tampering or malicious activity occurring on the system. This is achieved by monitoring log files, users, and the file system. Host based can use system logs, application logs, host traffic, and in some instances firewall logs as its data source. Some of activities that Host based can monitor include:

Ability to monitor user specific actions:
Host based can monitor the file system for file permission changes, privilege escalation, and watch certain users. Any changes that can happen, you would be notified right away, some even have the ability to prevent these attacks from ever occurring.
Access to system log files, running processes, and files system:
Host based ID systems have the capability to watch system log files and search for certain strings/patterns and generate an alarm.
Ability to determine the success/failure of an attack:
Since Operating Systems log what events have occurred, it makes it extremely easier to determine the success/failure of an attack. This makes the rate of false positives reduced.
Attacks that use NIDS evasion techniques:
Many tools used by intruders come with NIDS evasion techniques built-in. Network Intrusion Detection Systems that aren't updated regularly might miss the new method of NIDS evasion. Host based will be able to log the attack, log either failure or success of the attack.
Host based intrusion detection go hand in hand with Network based. The more information that is gathered during/after an attack can greatly increase the administrator's chances of identifying the source, type of attack, and hopefully thwarting further attacks.

Network Based:

Network based Intrusion Detection Systems (NIDS) can monitor both ingress and egress traffic. There are two forms of NIDS, Pattern Matching and Anomaly based. NIDS use network traffic as its daa source; monitoring network traffic in real time, and alerting in near real time.
Live Network Traffic:
The capability to use live network traffic as the data source reduces the chances of tampering, ensuring that what is captured is what is seen on the wire.
Detection of Attacks at Time of Occurrence:
Unlike their counterparts, network based can notify you the instant an attack was noticed. The faster an administrator is notified, faster they can respond. With faster notification, it can reduce the damage caused if the attack was successful.
Detect Unsuccessful Attacks:
The capability to detect unsuccessful attacks can bring light to malicious intent. The discovery of unsuccessful attacks can lead to notification of further attempts from the attacker.
As discussed above the basic level of intrusion detection systems have two forms: host and network based. Network based itself has two1 in two forms: Pattern Matching and Anomaly Baed, which are briefly discussed below.
Pattern Matching:
Most intrusion detection systems are pattern matching based systems. The intrusion detection system contains prior information about specific attacks and vulnerabilities. It appliesthis to ingress and egress traffic by inspecting each packet against its signature database. When such a condition is met, an alarm is triggered and the administrator is notified. The accuracy of a Knowledge based system relies on its signature databases.
Anomaly Based:
Anomaly based intrusion detection system creates a profile of normal network traffic. Once the profile of the current network is constructed any anomalous/irregular traffic that is seen will be considered suspicious (e.g. SSH traffic on port 4000), thus an alarm is generated. Detection of suspicious events can be implemented various ways, RFC compliance checking, Protocol analysis/decoding, traffic doesn't comply with normal traffic criteria. The fact that protocols are well defined makes the use of Protocol Analysis a strong contender, but the fact that various protocols fail to follow their respective RFC; this can also be a disadvantage.
Passive and Reactive Systems:
Host and Network based systems can either be passive systems or reactive based systems. Most network-based systems are passive with reactive capabilities. Passive systems detect possible attacks, log the information and issue an alert. Reactive systems attempt to react in some way to the malicious content it has spotted. Though reactive systems implement nice defensive mechanisms, they are still prone to false positives.

Reactive Network Based:

Reactive network based systems have the ability to react while watching the network, instead of a per system basis. This gives it authority to be reactive for a wide range of systems. More control per one intrusion detection system. Different methods of preventing/reacting are:
Ability to prevent known network/host based attacks from occurring.
Network based systems with reactive features have the capabilities to prevent attacks from occurring using various methods including, sending RST packets to reset the connection, dropping of offending packets/connections and various ICMP error codes.
Insertion of Firewall rules
Reactive systems can add offending source IP addresses into the deny/block chain of a firewall. This is limited to what firewall systems the intrusion detection system can understand. This can be useful if your IDS are seeing malicious packets from one source IP address.
Packet Scrubbing
When the packet passes the IDS it will inspect it with what it's configured to look for, if parts of the packet are malicious this methods has the ability to rewrite the offending prt to something non-malicious. For example in a packet sent by an exploit for certain vulnerability, the packet scrubber can rewrite/remove the /bin/bash portion of the payload, thus complete exploitation will fail.
Each method is part of the intrusion detection system thus is prone to false positives which might deny valid users/clients network connectivity. Some methods are permanent; others are a temporary solution or even per packet solutions.

Reactive host based:

Events are entered into log files after completion, thus to rely on reading log files for reactive tactics won't work. Reactive host based systems will tend to watch the actual filesystem, kernel and various other components for malicious/illegal content.

Improper privilege escalation

While watching system calls and the kernel, an attempt to escalate privileges can be seen, a reactive host based IDS can attempt to thwart this by killing the process.

Logging off malicious users

If activity is encountered that appears to be malicious, a reactive system can log the offending user off the system and block him from accessing the system until further notice posibly from an administrator of that host.

Malicious code execution

This would require a method other than watching log files. The ability to watch the kernel gives the reactive host based system the ability to identify and thwart malicious code exeution before the desired effect can occur.

Last words:

Keeping networks secure is no easy task. Managed services and 24x7 operations exist in an attempt to keep client networks secured and even then it's still no easy task. Having a multi-layered scurity infrastructure is a great start and now you can add intrusion detection to that list. Proactively keeping your IDS up to date is necessary and worthwhile for the survival of your network's security.

I included some project links that achieve some of what I mentioned above.

Intrusion Detection:
Snort Prelude2
Reactive Network based:
Hogwash SnortSam
Reactive Host based:
St. Jude3 Grsecurity
Alberto Gonzalez is Founder of Violating Networks. He contributes to various open source projects including The Bait N Switch Honeypot and Hogwash. He is currently in the process of getting hisGCIA certification from SANS.