Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Intrusion Detection System(IDS):
Recommended Papers

Nikolai Bezroukov Architectural Issues of Intrusion Detection Infrastructure in Large Enterprises

Nikolai Bezroukov "Slightly Skeptical View of Snort"

USENIX ;login - Table of Contents

Special Issue: Intrusion Detection (Sept. 1999) Table of Contents IN THIS ISSUE . . .

CONFERENCE REPORTS


Reports on the First Conference on Network Administration Reports on the Workshop on Intrusion Detection and Network Monitoring

FEATURES


Cisco Flow Logs and Intrusion Detection at the Ohio State University by Steve Romig, Mark Fullmer, and Suresh Ramachandran Invisible Intruders: Rootkits in Practice by David Brumley Indicators of UNIX Host Compromise by Paul C. Brutch, Tasneem G. Brutch, and Udo Pooch A Hacker's Approach to ID by Mudge A Glimpse Into the Future of ID by Tim Bass and Dave Gruber On Reliability by John Sellens

ANNOUNCEMENTS AND CALLS


3rd Annual Linux Showcase 3rd Large Installation System Administration of Windows NT / 2000 Conference 9th USENIX Security Symposium

Daemon News 199905 Intrusion Detection Systems

For those who are interested, a more in-depth analysis of these products can be found elsewhere.

ISS RealSecure by Internet Security Systems is probably the most well known intrusion detection system in the market. Its operation is similar to the Network Security Monitor mentioned earlier: it is connected to the network and listens to all the traffic passing through, searching for matches against the patterns it is configured to look for. It can monitor TCP, UDP and ICMP traffic, and in case a match is found countermeasures can be implemented. The announced bundling of the product with CheckPoint's Firewall-1 should become one of the best-selling security products in the market.

CyberCop by Network Associates is another product that follows the NSM paradigm. Its operation follows that of ISS RealSecure in the detection level, but its architecture is a more distributed one. The system is composed of a number of sensors that are scattered among the network nodes and a management server that collects intrusion reports sent by the sensors. In case an intrusion is detected the management server presents an elaborate but concise description of the event to the security manager, who can deal with the problem. CyberCop has the advantage to be backed by one of the largest companies in software business today and especially in the field of security.

Bro is an IDS developed at the Lawrence Berkley National Laboratory. Its source code is freely available and the architecture on which it is based is a modular one. The event engine is separated from the policy script interpreter, which dictates the policy implemented through a proprietary language. The event engine is designed to be capable of monitoring network connections of more than 100 Mbps, indicating that performance has been emphasized throughout its design.

NID is another freely available IDS. It is installed on a dedicated system from where it monitors network traffic. It searches for known attack signatures, as well as deviations from normal behavior inside the network. In case an intrusion is detected the security manager is notified.

USENIX ;login - intrusion-detection systems by Dario Forte

Generally speaking, IDSs are grouped into two families: the first performs mainly auditing functions, carrying out continuous calls to the hosts linked to the system. The second works independently, observing the traffic on the networks directly (and passively), carrying out "usual" packet filtering. Some commercial products combine the two methods.

Many IDSs feature automatic network-attack recognition and response-system type. The IDSs are installed and made to "run" throughout the network or on parts of it where there is a need to preserve critical information.

Monitoring of data traffic generally takes place on the TCP/IP stream passing through Ethernet-based infrastructures. (Some manufacturers are working on supports for different layouts.) Thus, the IDS combines the functions of a simple network analyzer with that of recognizing attacks. This is generally achieved by means of a signature check, rather like programs that detect viruses. The system contains a database of attacks, which functions as a basis of comparison with the traffic analyzed. When an attack is recognized, the IDS can stop the stream of data immediately, thereby preventing the attack

Arguments about the Effectiveness of IDSs

To what extent can an IDS influence the performance of a network? How effective can it really be? Management wonders most about such issues. Two well-known American security experts, Thomas T. Pracek and Timothy N. Newsham, have recently triggered a discussion about the true effectiveness of IDSs. According to these two experts, these types of device have many "weak points." However, most of these Achilles' heels reside in the analysis models.

In considering IDSs based on the signature-analysis method, also known as misuse detection, Pracek and Newsham have raised the objection that often the policies implemented are too stiff and are linked to mere signature matching; that is, in practice they are based excessively on passive monitoring. Errors made during configuration can lead to a burst of false alarms, particularly when specific services are used. To this end, the two experts, who state that they are more in favor of the active-proxy monitoring type of setup, feel that maximum granularity of the IDSs based on this method is required. As they have pointed out, highlighting the possible leaks in the analysis and operational model of an IDS does not label these devices as irreparably unsafe -- not at all. Instead, it points to a series of improvements to make to products in order to make these systems as safe as possible.

[PDF] THE INTEGRATION OF SECURITY SENSORS INTO THE INTELLIGENT INTRUSION Master thesis

Cisco Flow Logs and Intrusion Detection at the Ohio State University

Intrusion Detection for Check Point FireWall-1

Lance Spitzner Last Modified: 23 May, 2000

Firewalls do a good job of keeping the bad guys out.  But wouldn't it be nice to know when the bad guys are knocking on your door?  This article covers just that, how to determine when the bad guys are probing your network.  We discuss how you can use the FW-1 User Defined Alert to track when you are being probed, and by whom.  If you would like to see actual intrusion detection results, click here.  NOTE: This script works only on Unix based systems. Download the latest version here.

Ports: The IDS script has been ported to run on NT systems. I have not personally tested any of the scripts, you are on your own. James Oryszczyn has developed detect.pl, written in PERL Oscar Wahlberg has developed fw1_alert.pl, written in PERL Roberson Andrew has written alert.zip, written mainly in Windows Script Host 5.1

New Features with ver 1.4.3: Removed TrackDown() function, due to changes in Whois database. Tested and runs on most Unix flavors, including Linux, Solaris, and Nokia

UNIX Review - Kernel Audit Security Data

Analysis and Response for Intrusion Detection in Large Networks

COAST Audit Trails Format Group

List of accepted papers

BW: Network ICE Offers First Intrusion Detection System for Linux(May 08, 2000) LinuxSecurity.com: Build a Secure System with LIDS(Apr 25, 2000) Lids.org: LIDS Hacking HOWTO(Apr 09, 2000) Network Computing: Best Practices in Network Security(Mar 18, 2000) LinuxSecurity.com: Intrusion Detection Primer(Mar 13, 2000) TechWeb: Linux Suppliers Focus On Improved Security [via Tripwire](Mar 03, 2000) Security Portal: Some thoughts on (network) intrusion detection systems(Jan 16, 2000) Security Portal: Network Intrusion Detection Systems and Virus Scanners - are they the answer?(Jan 09, 2000) Security Portal: Do you have an Intrusion Detection Response Plan?(Aug 24, 1999) Security Portal: Detecting Intruders in Linux(Aug 16, 1999)

 

Slashdot: Intrusion Detection [Book Review](Jan 27, 2000)

Slashdot: Review: Network Intrusion Detection: An Analysis Handbook(Sep 28, 1999)

http://www.anticode.com - huge selection, nicely layed out, the supermarket of exploit sites http://www.rootshell.com - out of date but nicely searchable index http://www.technotronic.com - exploit code and other tools such as sniffers and intrusion detection http://www.nmap.org - Nmap scanner http://www.nessus.org - Nessus intrusion scanner http://www.marko.net/cheops/ - Cheops http://www.isag.com - Internet Security Advisors Group (Ira Winkler is President of said company) http://securityportal.com/research/research.underground.html - Underground Resources

Summer Dreams of IDS by Rik Farrow

First, some terminology. There are at least two general types of IDS: host-based and network-based. Host-based systems are installed on the server or desktop they're designed to protect, and generally monitor logfiles for certain events and key files for changes. Some host-based systems are hybrids, as they also monitor network traffic sent to the host where they're installed. Host-based IDSs send alerts to a central console, as do network-based systems.

Network-based IDSs (NIDSs) sniff network traffic using a system called a sensor. The sensor collects all packets and evaluates both the network headers and the data, looking for signs of misuse. Many sensors focus on attack signatures-that is, a pattern in the header or data that matches a known attack. Some sensors go beyond mere signature matching, attempting to match traffic with correct layer-4 and layer-7 protocols.

... ... ...

I recently spoke with an employee of a large Web hosting and Internet services company about NIDS' practical uses. This employee uses the free NIDS tool snort to monitor the internal backbone at a large colocation facility. At this facility, snort collects between five thousand and six thousand alerts each weekday (fewer over weekends), and this is with a trimmed list of attack signatures. With so many potential incidents, all they do at this site is filter through the alert logs on an hourly basis to determine exactly what is significant and what, if anything, can be done about it.

Most sites won't have to deal with thousands of alerts each day. But even dealing with several alerts each day is a full-time job for someone skilled in network and operating system security. The person monitoring the IDS console must first perform triage-deciding which alert deserves immediate attention, and which can wait. Then this person must locate the potentially compromised system, determine if the attack has succeeded, and, if so, decide the next step. Should the system be further analyzed? Saved for evidence? Or just reformatted, with a new, patched operating system and applications installed? And does this attack hint at more undiscovered vulnerabilities on other systems that should also be patched immediately?

Daemon News 199905 Intrusion Detection Systems

For those who are interested, a more in-depth analysis of these products can be found elsewhere.

ISS RealSecure by Internet Security Systems is probably the most well known intrusion detection system in the market. Its operation is similar to the Network Security Monitor mentioned earlier: it is connected to the network and listens to all the traffic passing through, searching for matches against the patterns it is configured to look for. It can monitor TCP, UDP and ICMP traffic, and in case a match is found countermeasures can be implemented. The announced bundling of the product with CheckPoint's Firewall-1 should become one of the best-selling security products in the market.

CyberCop by Network Associates is another product that follows the NSM paradigm. Its operation follows that of ISS RealSecure in the detection level, but its architecture is a more distributed one. The system is composed of a number of sensors that are scattered among the network nodes and a management server that collects intrusion reports sent by the sensors. In case an intrusion is detected the management server presents an elaborate but concise description of the event to the security manager, who can deal with the problem. CyberCop has the advantage to be backed by one of the largest companies in software business today and especially in the field of security.

Bro is an IDS developed at the Lawrence Berkley National Laboratory. Its source code is freely available and the architecture on which it is based is a modular one. The event engine is separated from the policy script interpreter, which dictates the policy implemented through a proprietary language. The event engine is designed to be capable of monitoring network connections of more than 100 Mbps, indicating that performance has been emphasized throughout its design.

NID is another freely available IDS. It is installed on a dedicated system from where it monitors network traffic. It searches for known attack signatures, as well as deviations from normal behavior inside the network. In case an intrusion is detected the security manager is notified.

USENIX ;login - intrusion-detection systems by Dario Forte

Generally speaking, IDSs are grouped into two families: the first performs mainly auditing functions, carrying out continuous calls to the hosts linked to the system. The second works independently, observing the traffic on the networks directly (and passively), carrying out "usual" packet filtering. Some commercial products combine the two methods.

Many IDSs feature automatic network-attack recognition and response-system type. The IDSs are installed and made to "run" throughout the network or on parts of it where there is a need to preserve critical information.

Monitoring of data traffic generally takes place on the TCP/IP stream passing through Ethernet-based infrastructures. (Some manufacturers are working on supports for different layouts.) Thus, the IDS combines the functions of a simple network analyzer with that of recognizing attacks. This is generally achieved by means of a signature check, rather like programs that detect viruses. The system contains a database of attacks, which functions as a basis of comparison with the traffic analyzed. When an attack is recognized, the IDS can stop the stream of data immediately, thereby preventing the attack

Arguments about the Effectiveness of IDSs

To what extent can an IDS influence the performance of a network? How effective can it really be? Management wonders most about such issues. Two well-known American security experts, Thomas T. Pracek and Timothy N. Newsham, have recently triggered a discussion about the true effectiveness of IDSs. According to these two experts, these types of device have many "weak points." However, most of these Achilles' heels reside in the analysis models.

In considering IDSs based on the signature-analysis method, also known as misuse detection, Pracek and Newsham have raised the objection that often the policies implemented are too stiff and are linked to mere signature matching; that is, in practice they are based excessively on passive monitoring. Errors made during configuration can lead to a burst of false alarms, particularly when specific services are used. To this end, the two experts, who state that they are more in favor of the active-proxy monitoring type of setup, feel that maximum granularity of the IDSs based on this method is required. As they have pointed out, highlighting the possible leaks in the analysis and operational model of an IDS does not label these devices as irreparably unsafe -- not at all. Instead, it points to a series of improvements to make to products in order to make these systems as safe as possible.

Cisco Flow Logs and Intrusion Detection at the Ohio State University

Intrusion Detection for Check Point FireWall-1

Lance Spitzner
Last Modified: 23 May, 2000

Firewalls do a good job of keeping the bad guys out.  But wouldn't it be nice to know when the bad guys are knocking on your door?  This article covers just that, how to determine when the bad guys are probing your network.  We discuss how you can use the FW-1 User Defined Alert to track when you are being probed, and by whom.  If you would like to see actual intrusion detection results, click here.  NOTE: This script works only on Unix based systems. Download the latest version here.

Ports:
The IDS script has been ported to run on NT systems. I have not personally tested any of the scripts, you are on your own.
James Oryszczyn has developed detect.pl, written in PERL
Oscar Wahlberg has developed fw1_alert.pl, written in PERL
Roberson Andrew has written alert.zip, written mainly in Windows Script Host 5.1

New Features with ver 1.4.3:
Removed TrackDown() function, due to changes in Whois database.
Tested and runs on most Unix flavors, including Linux, Solaris, and Nokia

EBN - Summer Dreams of IDS

By Rik Farrow
08/03/2001 10:16 AM EST
URL: http://www.itarchitect.com/shared/article/showArticle.jhtml?articleId=8703157

Imagine network security the way it should be. You show up for work in the morning and get a report via e-mail from your Intrusion Detection System (IDS): Everything is OK. You might feel a little dubious about this, but why should you? After all, isn't the goal of network security to prevent problems in the first place? resou Today's IDSs don't come close to reaching that goal. Instead, their main allure is reporting what might be wrong in your network, and most products focus on producing lots of alerts about incidents so that you don't feel that you wasted your time and money deploying the system. And while knowing what's happening on your network is a worthwhile goal, it doesn't in itself do anything to solve the problems manifesting through the medium of the IDS console.

Perhaps I am being a bit unfair. But if you do nothing but listen to vendors rave about their products, you are bound to be misled and disappointed when the reality of what your shiny new IDS can't do sets in. IDS products are getting better, but they still have a long way to go.

A Distant Dream?

This article is not about products. Producing a useful review of IDS products is a very difficult task-one that's illustrated well by a review by Greg Shipley, director of consulting services with Neohapsis

(see Resources). According to Shipley, the review should be published at roughly the same time as this issue of Network Magazine.

In this column, I'll focus on the technology behind IDSs, as well as an idealized view on some features that would dramatically increase these products' functionality.

First, some terminology. There are at least two general types of IDS: host-based and network-based. Host-based systems are installed on the server or desktop they're designed to protect, and generally monitor logfiles for certain events and key files for changes. Some host-based systems are hybrids, as they also monitor network traffic sent to the host where they're installed. Host-based IDSs send alerts to a central console, as do network-based systems.

Network-based IDSs (NIDSs) sniff network traffic using a system called a sensor. The sensor collects all packets and evaluates both the network headers and the data, looking for signs of misuse. Many sensors focus on attack signatures-that is, a pattern in the header or data that matches a known attack. Some sensors go beyond mere signature matching, attempting to match traffic with correct layer-4 and layer-7 protocols.

While researching this column, I spoke with Shipley, who disclosed some results of the Neohapsis product review. Some of the details were striking. For example, many NIDS sensors crashed during the testing under real, not test, network loads. The fastest sensor caught almost every attack, but it had a user interface that only a Unix sysadmin would adore. A different product with an even faster sensor had a great user interface, but only a tiny signature database-so it missed many attacks.

Another issue for existing NIDS is that some of the best-known vendors are only now catching up to some of the tricks used to evade IDSs. In early 1998, Thomas Ptacek and Timothy Newsham published a paper about bypassing NIDSs using fragmentation and out-of-order packets. And what was true in 1998 remains true to some degree today. While many NIDS do attempt to defragment packets sniffed off a network, it's just not as simple as that. (See Network Defense, July 1999, for more information on fragmentation issues and NIDS.) And while vendors might claim it's not so easy to fragment attacks, Dug Song, while an employee of IDS vendor Anzen, wrote fragrouter, a tool that makes fragmenting any network traffic easy.

A related issue has to do with out-of-order packets. Another way of playing hob with NIDS is sending small packets that contain an attack split among them, so that the attack signature doesn't appear in any one of them, and sending the first packet last. The NIDS sensor's task is to reassemble the data stream, then analyze it for attacks in what IDS vendors now call "maintaining state." But adding this detail to an NIDS sensor already operating at its performance limits often pushes it right over the edge.

A sensor claiming to monitor 100Mbits/sec can often only do this part of the time. In another article (see Resources), Shipley claims that this is due in part to packet size. A 100Mbit/sec Ethernet link filled with maximum-size packets can carry, at most, 8,120 packets per second. But send the smallest possible packets (for example, pure TCP ACK packets), and the packets-per-second rate goes up to 144,880. Now, to make things really interesting, let's make many of those packet acknowledgments to ongoing TCP connections that the sensor is supposed to be monitoring in its state table-and you can really begin to understand why many sensors experience meltdown.

It's one thing to sniff 144,880 packets per second (no mean feat), and another to check them for known attack signatures. Matching them to existing data streams before checking them, however, is quite another matter. An attacker can, through a port scanner, quickly set up thousands of connections in the state table, leading to memory overload. According to Shipley, many products he reviewed had memory leaks-places in the code where memory was used but never freed-that led to crashes over time.

People Power

IDSs' failings may be legion, but I'm not claiming these products are useless. In fact, a properly working IDS brings up the next big issue for proper deployment: Who will handle the potential security incident that the IDS reports?

I recently spoke with an employee of a large Web hosting and Internet services company about NIDS' practical uses. This employee uses the free NIDS tool snort to monitor the internal backbone at a large colocation facility. At this facility, snort collects between five thousand and six thousand alerts each weekday (fewer over weekends), and this is with a trimmed list of attack signatures. With so many potential incidents, all they do at this site is filter through the alert logs on an hourly basis to determine exactly what is significant and what, if anything, can be done about it.

Most sites won't have to deal with thousands of alerts each day. But even dealing with several alerts each day is a full-time job for someone skilled in network and operating system security. The person monitoring the IDS console must first perform triage-deciding which alert deserves immediate attention, and which can wait. Then this person must locate the potentially compromised system, determine if the attack has succeeded, and, if so, decide the next step. Should the system be further analyzed? Saved for evidence? Or just reformatted, with a new, patched operating system and applications installed? And does this attack hint at more undiscovered vulnerabilities on other systems that should also be patched immediately?

Just installing a single sensor and an NIDS console can easily consume weeks. Proper installation involves not only the physical process of installing the sensor and the console software, but also configuring the NIDS itself. Almost all IDS products support some level of customization. Sometimes the IDS will do this "automatically"-you launch a tool, and the tool scans the network, attempts to identify operating systems, then only watches for attacks specific to those IP addresses and the operating system associated with a particular address. Of course, you must remember to launch this configuration process anytime a system is added to your network.

More than likely, you'll need to manually select sets of attack signatures that you're most interested in. For example, if your organization has nothing but Microsoft systems, you don't want the IDS console annoying you with alerts about Unix/Linux exploits directed at internal systems. You might want the IDS to alert you to attacks against external Unix systems coming from your own network, though. And as you use a tool, you'll begin to learn which alerts will most likely be false positives-something the IDS picks up as an attack, but which actually represent normal network activity at your site.

The console will also be of critical importance to users. It must be capable of providing a high-level view but still permit you to drill down to collect more detailed information about a particular alert. There must be a mechanism for clearing alerts when you're done with them, or annotating the alerts, so you can remember later what you've done about a specific alert.

Hopefully, the console won't just present you with a terse log-style message, but will provide information about what the alert means, how serious it might be, how to actually verify if the alert represents a real event and, finally, how to patch the system involved.

Keeping the Dream Alive?

While some IDSs do hint at what an alert means, and a few even make suggestions about how to check a system and see if the attack has been successful, that's far from the "dream" IDS envisioned earlier in this column. But let's stay with that dream for a minute and imagine what a Manhattan Project approach ($2 billion in 1945 dollars) could do to implement the ideal IDS.

The ideal IDS would be a hybrid, with both host-based and network-based sensors. The host-based module would also permit operating system software and upgrading on monitored machines, and the ability to adjust file permissions/ownerships, registry keys/configuration, and operating system tuning parameters-all without rebooting. And this host-based IDS would consume few CPU cycles, so installing it even on heavily loaded servers wouldn't be a problem.

The NIDS sensors would be installed in switches (two vendors have already achieved this) so that all traffic could easily be monitored. The sensors wouldn't just be passive network traffic monitors, but would keep track of in-use IP addresses and use passive fingerprinting to identify operating systems, and active fingerprinting if necessary. If a new system appeared, the sensor would go into active mode and perform a vulnerability scan of the new system, installing any operating system or application updates that were previously approved for that platform. The sensor would also inform the central console of a new system on the network.

The NIDS sensor would keep a history of all network traffic. If a new service suddenly appeared on a system, this could be an indication of either a successful attack or the installation of a Trojan. The NIDS would not only alert the console but also might take countermeasures against the possible Trojan-especially if the NIDS sensor detected other suspicious activity, such as a sudden flood or port scan originating from the suspect system.

While this type of system remains a dream, it's not mine alone. Department of Defense (DoD) research projects such as Emerald and Sapphire focus on assimilating reports from an array of sensors to provide the big picture, correlation between reports, and the ability to drill down to get details from individual sensors or hosts. Someday, the dream may come true.

Rik Farrow is an independent security consultant. His Web site, www.spirit.com, contains security links and information about network and computer security courses. He can be reached at rik@spirit.com.


Resources

Thomas Ptacek and Timothy Newsham's Intrusion Detection (ID) paper is located at this link.

Greg Shipley's article about Intrusion Detection Systems (IDSs) from Network Computing is located here.

Greg Shipley and Neohapsis' review of ID products will appear at Neohapsis sometime in August 2001.

Dug Song wrote fragrouter, a tool for testing network IDSs, while he worked for Anzen. Go to http://packetstorm.securify.com/UNIX/IDS/fragrouter-1.6.tar.gz or www.monkey.org to download fragrouter.

The CERIAS page defines IDS terms and provides IDS references. Go to this link.

A nice summary of different types of IDSs (including vulnerability scanners) can be found here.

The Evolution of Intrusion Detection Systems

Paul Innella 2001-11-16

The Evolution of Intrusion Detection Systems
by Paul Innella, Tetrad Digital Integrity, LLC
last updated November 16, 2001

Introduction

I am currently working with a client who asked me to choose an intrusion detection system (IDS) to deploy in their environment. I have been working with intrusion detection since it was virtually unknown, so it would seem the decision would be quite simple. On the contrary, with all of the different components and vendors to choose from, IDS offerings have become pretty complex. That led me to wonder how IDS technology has progressed to its current state. So, I invested some time trying to figure it out. Now that I have, let me tell you, it is enough to induce a headache. Nonetheless, I wrote this article to share my findings with you. If you are ready for a discussion about the evolution of IDS, then read on; however, be forewarned, the history of intrusion detection is as confusing as Greenspan's economic strategies.

IDS Components

Before we get started, let me provide a layman's description of the primary IDS components:

Network Intrusion Detection (NID)

Network intrusion detection deals with information passing on the wire between hosts. Typically referred to as "packet-sniffers," network intrusion detection devices intercept packets traveling along various communication mediums and protocols, usually TCP/IP. Once captured, the packets are analyzed in a number of different ways. Some NID devices will simply compare the packet to a signature database consisting of known attacks and malicious packet "fingerprints", while others will look for anomalous packet activity that might indicate malicious behavior. In either case, network intrusion detection should be regarded primarily as a perimeter defense.

NID has historically been incapable of operating in the following environments:

  1. Switched networks
  2. Encrypted networks
  3. High-speed networks (anything over 100 Mbps)

Recently, however, Cisco released a module for their Catalyst 6000 switch that incorporates network intrusion detection directly in the switch, overcoming the first of these flaws. Additionally, ISS/Network ICE indicated that they are now capable of "packet-sniffing" at gigabit speeds.

Host-based Intrusion Detection (HID)

Host-based intrusion detection systems are designed to monitor, detect, and respond to user and system activity and attacks on a given host. Some more robust tools also offer audit policy management and centralization, supply data forensics, statistical analysis and evidentiary support, and in certain instances provide some measure of access control. The difference between host-based and network-based intrusion detection is that NID deals with data transmitted from host to host while HID is concerned with what occurs on the hosts themselves.

Host-based intrusion detection is best suited to combat internal threats because of its ability to monitor and respond to specific user actions and file accesses on the host. The majority of computer threats come from within organizations, from many different sources; disgruntled employees and corporate spies are just two examples. In fact, intrusion detection expert Richard Power states, "each year, we've asked the respondents [of the CSI/FBI survey] to rate the likely sources of [a network] attack. Invariably, disgruntled and dishonest employees have topped the list, with over 80% of respondents citing them as a likely source." (Power, 1999: 32.)

Hybrid Intrusion Detection

Hybrid intrusion detection systems offer management of and alert notification from both network and host-based intrusion detection devices. Hybrid solutions provide the logical complement to NID and HID - central intrusion detection management.

Network-Node Intrusion Detection (NNID)

Network-node intrusion detection was developed to work around the inherent flaws in traditional NID. Network-node pulls the packet-intercepting technology off of the wire and puts it on the host. With NNID, the "packet-sniffer" is positioned in such a way that it captures packets after they reach their final target, the destination host. The packet is then analyzed just as if it were traveling along the network through a conventional "packet-sniffer." This scheme came from a HID-centric assumption that each critical host would already be taking advantage of host-based technology. In this approach, network-node is simply another module that can attach to the HID agent. Network node's major disadvantage is that it only evaluates packets addressed to the host on which it resides. Traditional network intrusion detection, on the other hand, can monitor packets on an entire subnet. Even so, "packet-sniffers" are equally incapable of viewing a complete subnet when the network uses high-speed communications, encryption, or switches since they are essentially "without a sense of smell" (first and last NID joke, I promise.) The advantage to NNID is its ability to defend specific hosts against packet-based attacks in these complex environments where conventional NID is ineffective.

Intrusion Detection Systems: A Brief History

The goal of intrusion detection is to monitor network assets to detect anomalous behavior and misuse. This concept has been around for nearly twenty years but only recently has it seen a dramatic rise in popularity and incorporation into the overall information security infrastructure. Beginning in 1980, with James Anderson's paper, Computer Security Threat Monitoring and Surveillance, the notion of intrusion detection was born. Since then, several pivotal events in IDS technology have advanced intrusion detection to its current state. Let's focus on how IDS has progressed since its inception.

James Anderson's seminal paper, written for a government organization, introduced the notion that audit trails contained vital information that could be valuable in tracking misuse and understanding user behavior. With the release of this paper, the concept of "detecting" misuse and specific user events emerged. His insight into audit data and its importance led to tremendous improvements in the auditing subsystems of virtually every operating system. Anderson's conjecture also provided the foundation for future intrusion detection system design and development. His work was the start of host-based intrusion detection and IDS in general.

In 1983, SRI International, and specifically Dr. Dorothy Denning, began working on a government project that launched a new effort into intrusion detection development. Their goal was to analyze audit trails from government mainframe computers and create profiles of users based upon their activities. One year later, Dr. Denning helped to develop the first model for intrusion detection, the Intrusion Detection Expert System (IDES), which provided the foundation for the IDS technology development that was soon to follow.

In 1984, SRI also developed a means of tracking and analyzing audit data containing authentication information of users on ARPANET, the original Internet. Soon after, SRI completed a Navy SPAWAR contract with the realization of the first functional intrusion detection system, IDES. Using her research and development work at SRI, Dr. Denning published the decisive work, An Intrusion Detection Model, which revealed the necessary information for commercial intrusion detection system development. Her paper is the basis for most of the work in IDS that followed.

Meanwhile, there were other significant advances occurring at University of California Davis' Lawrence Livermore Laboratories. In 1988, the Haystack project at Lawrence Livermore Labs released another version of intrusion detection for the US Air Force. This project produced an IDS that analyzed audit data by comparing it with defined patterns. In a telephone interview with the author, Crosby Marks, a former Haystack Project team member and Haystack Labs employee said that, "searching through this large amount of data for one specific misuse was equivalent to looking for a needle in a haystack."

The subsequent iteration of this tool was called the Distributed Intrusion Detection System (DIDS). DIDS augmented the existing solution by tracking client machines as well as the servers it originally monitored. Finally in 1989, the developers from the Haystack project formed the commercial company, Haystack Labs, and released the last generation of the technology, Stalker. Crosby Marks says that "Stalker was a host-based, pattern matching system that included robust search capabilities to manually and automatically query the audit data." The Haystack advances, coupled with the work of SRI and Denning, greatly advanced the development of host-based intrusion detection technologies.

To kick off the '90s, UC Davis's Todd Heberlein introduced the idea of network intrusion detection. In 1990, Heberlein was the primary author and developer of Network Security Monitor (NSM), the first network intrusion detection system ( see Heberlein, L. et al. "A Network Security Monitor." Proceedings of the IEEE Computer Society Symposium, Research in Security and Privacy, May 1990, pp. 296-303.) NSM was deployed at major government installations where network traffic analysis provided massive amounts of information. This new awareness generated more interest in the field of intrusion detection and investments in that market increased significantly. Heberlein's contributions also extended to the DIDS project where, along with the Haystack team, he introduced the first idea of hybrid intrusion detection. The work of the Haystack project and the introduction of the Network Security Monitor revolutionized the IDS field and brought it into the commercial world.

Commercial development of intrusion detection technologies began in the early 1990s. Haystack Labs was the first commercial vendor of IDS tools, with its Stalker line of host-based products. SAIC was also developing a form of host-based intrusion detection, called Computer Misuse Detection System (CMDS). Simultaneously, the Air Force's Cryptologic Support Center developed the Automated Security Measurement System (ASIM) to monitor network traffic on the US Air Force's network. ASIM made considerable progress in overcoming scalability and portability issues that previously plagued NID products. Additionally, ASIM was the first solution to incorporate both a hardware and software solution to network intrusion detection. ASIM is still currently in use and managed by the Air Force's Computer Emergency Response Team (AFCERT) at locations all over the world. As often happened, the development group on the ASIM project formed a commercial company in 1994, the Wheel Group. Their product, NetRanger, was the first commercially viable network intrusion detection device. Nonetheless, commercial intrusion detection systems developed slowly during these years and only truly blossomed towards the latter half of the decade.

The intrusion detection market began to gain in popularity and truly generate revenues around 1997. In that year, the security market leader, ISS, developed a network intrusion detection system called RealSecure. A year later, Cisco recognized the importance of network intrusion detection and purchased the Wheel Group, attaining a security solution they could provide to their customers. Similarly, the first visible host-based intrusion detection company, Centrax Corporation, emerged as a result of a merger of the development staff from Haystack Labs and the departure of the CMDS team from SAIC. From there, the commercial IDS world expanded its market-base and a roller coaster ride of start-up companies, mergers, and acquisitions ensued. (The next section, Players, discusses these developments.)

Currently, market statistics show that IDS is amidst the top selling security vendor technologies and should continue to rise. Furthermore, government initiatives, such as the Federal Intrusion Detection Network, (FIDNet) created under Presidential Decision Directive 63, are also adding impetus to the evolution of IDS. Advancements in IDS will ultimately push security technology into a whole new arena of automated security intelligence.

So what's next? Well, there's application intrusion detection, heuristics and rules-based intrusion detection, incorporation of artificial intelligence, and who knows what else. (If I did I would be working on that instead of writing this article.) Regardless of how intrusion detection technology evolves, one thing is for sure - it is now an important and integral component of information security.

Players

The IDS market appears to grow and shrink with the stock market. In the February 2000 timeframe there seemed to be a gaggle of IDS vendors, but there are now only a few serious ones remaining. Most have faded away or been gobbled up by the bigger fish. It is important to note how different intrusion detection vendors' technologies came to be. So, let's review the current players in the market and discuss how they got here.

Cisco

Cisco currently provides both host-based and network-based intrusion detection products. They first entered the IDS market in 1997 by acquiring the Wheel Group and their NetRanger technology for $124 million. Their IDS 4230 and 4210 series provide typical NID solutions while the Catalyst 6000 module is the first switched-integrated NID offering on the market. On the host-based intrusion detection side, they use Entercept's technology instead of having developed their own.

Internet Security Systems (ISS)

ISS provides both a proprietary hybrid intrusion detection solution and an additional NID system that functions independently of their hybrid offering. ISS ventured into NID with the release of RealSecure, virtually at the same time that CISCO purchased NetRanger. Only two years later, they introduced a host-based component that made their hybrid IDS offering complete. ISS recently made another move towards conquest of the IDS market with the acquisition of Network ICE and their highly respected NID solutions, including their new gigabit NID system.

Symantec

Symantec recently acquired Axent and thus gained their Intruder Alert and NetProwler technologies, bringing them into the IDS market. Symantec offers both NID and HID solutions and a method of providing hybrid integration as well.

Enterasys

Like Cisco, Enterasys/Cabletron realized that providing bridges and routers was a viable means of subsequently offering intrusion detection. Consequently, Enterasys purchased Network Security Wizards and their Dragon NID solution and Squire HID system.

The Rest

Now, letís discuss the rest of the IDS field. Okay, ready, take a deep breath, and here we go.

We start with the merger of the CMDS development staff from SAIC who went on to start Centrax Corporation along with the people from Haystack Labs. Incidentally, the remainder of Haystack was purchased by Trusted Information Systems, which was then acquired by Network Associates. Centrax Corporation was purchased by CyberSafe Corporation and changed the host-based product, Entrax, into Centrax. Centrax then became the first commercially viable host-based intrusion detection product. CyberSafe later introduced NID followed by NNID technologies into their Centrax product; they even partnered with Network ICE to round out the solution - until ISS bought Network ICE and ended that relationship.

Meanwhile, CMDS moved to its new home when SAIC sold it to ODS, who renamed it to Kane Security Enterprise and then spun off Intrusion.com to better market the product. Intrusion.com then purchased MimeStar, and its SecureNet Pro NID solution, to provide its customers with a total IDS offering.

Okay, you can let it out now - the long and short of this series of spin-offs, acquisitions, mergers, and failures is that the strong seem to get stronger and the number of IDS competitors continue to decrease.

Conclusion

Summing up, the work of Anderson, Heberlein, and Denning spawned the concept of IDS. Government funding and corporate interest helped to develop their concept into a tangible technology that eventually found its way into the mainstream of network security. Intrusion detection has indeed come a long way, becoming a necessary means of monitoring, detecting, and responding to security threats. From theory to practice, and finally to commercially viable tools, IDS technology has gone through countless iterations and numerous owners. Nonetheless, the use of intrusion detection as a means of deterring misuse has ultimately become commonplace. Moreover, IDS has become essential.

References

Allen, J., Christie, A., Fithen, W., McHugh, J., Pickel, J., and Stoner, E., State of the Practice of Intrusion Detection Technologies. CMU/SEI-99-TR-028, Carnegie Mellon University, Software Engineering Institute, January 2000.

Anderson, James P., Computer Security Threat Monitoring and Surveillance. James P. Anderson Co., Fort Washington, Pa., 1980.

D. E. Denning, "An intrusion-detection model." IEEE Transactions on Software Engineering, Vol. SE-13(No. 2):222-232, Feb. 1987.

Heberlein, L. et al. "A Network Security Monitor." Proceedings of the IEEE Computer Society Symposium, Research in Security and Privacy, May 1990, pp. 296-303.

Marks, Crosby, former Haystack Project team member and Haystack Labs employee, telephone interview, September 3, 2001.

Mchugh, J. et al. "Intrusion Detection: Implementation and Operational Issues," Software Engineering Institute Computer Emergency Response Team White Paper, January 2001.

Power, Richard, "1999 CSI/FBI Computer Crime and Security Survey," Computer Security Journal, Volume XV, Number 2, 1999, pp. 32.

Proctor, Paul, The Practical Intrusion Detection Handbook, Prentice Hall, 2001.

Sans Institute, "Intrusion Detection and Vulnerability Testing Tools: What Works" Feb 2001.

Shipley, Greg, "Watching the Watchers: Intrusion Detection," Network Computing, November 13, 2000.

Paul Innella, CISSP, is President of Tetrad Digital Integrity, LLC a Washington, D.C. based information security services company. Mr. Innella has nearly ten years of experience in the computer industry working at several commercial and government companies as a security engineer, developer, integrator, systems administrator, program manager, and sales engineer. Mr. Innella is a member of the Gerson Lehrman Group Council of Advisors, the CISSP Speakers/SME Bureau, ISSA, and CSI.

 

SecurityFocus accepts Infocus article submissions from members of the security community. Articles are published based on outstanding merit and level of technical detail. Full submission guidelines can be found at http://www.securityfocus.com/static/submissions.html.

     



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Haterís Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019