May the source be with you,
but remember the KISS principle ;-)
Key Softpanorama Topics
|About||Contents||Top Updates||Top Visited|
|Bulletin||Selected Papers||Softpanorama Bookshelf||History|
|News||A Slightly Skeptical View on Snort||Network-level Intrusion Detection||Recommended Books||Recommended Links||Recommended Papers|
SHADOW (Secondary Heuristic Analysis for Defensive Online Warfare) is a very simple combination of tcpdump and Perl that actually works as a rudimentary NIDS but can made more powerful by using snort and/or custom Perl processing scripts. Due to flexibility and power of Perl the architecture used by Shadow remains one of the best NIDS architectures and in my opinion is architecturally superior to Snort.
The original Shadow code was developed by the US Navy as the result of the "Cooperative Intrusion Detection Evaluation and Response" (CIDER) project. Currently SHADOW codebase is unmaintained abandonware and the code base is largely obsolete. The latest published version (2003) was developed by NSWC.
Solaris is an excellent platform to run any NIDS and Shadow is not an exception. But most steps actually are OS independent.
Shadow basically consists of two parts that can run on different boxes (although it is not strictly necessary):
While the idea is transparent and very good, the code supplied is not very transparent and is of limited usefulness for any knowledgeable Perl programmer. There are things that definitely can be simplified.
The key files are:
There is also pretty convoluted file (sensor_driver.pl) to launch tcpdump and it probably can be discarded and replaced with a simple ksh script.
Actually you need is the following simple loop that can be invoked from cron each selected sensor using preselected interval, for example 15 min:
1. Write file with tcpdump to the archive (possibly compress it if you nee to work with huge pipe).
2. Optionally transfer it to the "analysis station".
3. Reprocess this file with TCPdump working in "listing mode", feeding the output to Perl analyzer that deciphers TCP dump format and produced actual alerts. Or better use Snort snort instead of TCPdump for doing this and reprocessed generated alerts with Perl (using Snort database format)
|Bulletin||Latest||Past week||Past month||
Title Build an IDS with Snort, Shadow, and ACID Date 2005.04.27 11:00 Author Mark Bain Topic
Every organization understands the importance of using a firewall to protect its assets. But what happens if someone finds a hole in the wall? What if the infiltrator is actually someone from within your organization who wants to access information that he shouldn't? To cover those contingencies, you need an intrusion detection system (IDS) to complement your firewall. Fortunately, with a minimum amount of time and money you can set up an IDS with open source tools such as Snort, Shadow, and ACID.
An IDS comprises sensors to do the actual monitoring and a central console to store and analyse the data collected on each sensor. A sensor can be a basic PC with an extra network card (to do the actual monitoring), whilst the central console should have a bit more processing power and a larger disk.
If Linux isn't already on the PCs you plan to use then you'll have to install it. You'll need only a minimal install because these boxes will be used only for the IDS. I recommend installing Bastille to make the IDS more secure. Another key thing to do is to put the monitoring network card on the sensor without an IP address; this makes it effectively invisible.
You can now start installing the software to do the actual intrusion detection. You can use Snort to monitor patterns of behaviour (or signatures) and Shadow to monitor packet headers. You'll need ACID to analyse the data collected by Snort (Shadow has its own analyser).
Let's begin with the sensor. The first piece of software to install is the client and develop RPMs for MySQL drivers, which are necessary because Snort store its output in a database on the central console. Once these drivers are in place, download the tar file for Snort, decompress it, and install it with the commands:./configure --with-mysql make make install
The installation and setup instructions for Snort are clear and concise. Snort uses a set of rules to identify signatures (or patterns of behaviour) to identify illegal activity. You can download a default set of customisable rules from Snort's site, along with some excellent documentation. If you have a Windows client, you can manage the signatures using Activeworx's IDS Policy Manager.
With Snort on the workstation, the next step is to install Shadow. The software comprises a set of Perl scripts that use tcpdump to collect data from packet headers on the network in order to identify any unusual activity. Shadow's Web site contains documentation that covers package dependencies (including Perl and tcpdump; the latter is supplied with Shadow), installation, and setup. To configure Shadow you need to modify three files -- sensor_init.sh, std.ph, and std.filter -- to reflect your local setup, such as the location of tcpdump. You also need to set up crontab to run sensor_driver.pl (the main script of the sensor) once an hour. The package supplies a suitable cron file -- /usr/local/SHADOW/sensor/sensor_crontab -- that also will synchronize the sensor time with a reliable Network Time Protocol source.
Finally, you must make sure that Shadow runs at system startup time by copying /usr/local/SHADOW/sensor_init.sh to your /etc/rc.d directory. Once Shadow is running, your sensor PC is ready to be connected to the network.
Shadow comes with its own analyser, which requires tcpdump, OpenSSH, and Apache. The Shadow documentation will lead you through the configuration process, tell you how to set up the Shadow CGI scripts, and explain how to set up crontab for automatic operation.
With the sensor set up you can turn to the central console, where the actual analysis of the data will be done.
Snort has no analysis software itself -- it simply generates data on the sensor and then stores it in a MySQL database on the central console. You can use ACID (Analysis Console for Intrusion Databases) to analyse the data. ACID's Web site gives full details on software installation and the creation of the MySQL database for storing the Snort alerts.
At this point your IDS should be up and running with Snort, Shadow, and ACID. On the sensor Snort will be monitoring the traffic patterns and raising alerts to the database. Shadow will be reading packet headers and creating log files. On the central console you will be able to view the Web pages created by Shadow, and to use ACID to analyse the alerts created by Snort.
Now the real work begins. You must learn how to understand the traffic on your own network so that you can filter the false alarms from the real ones. To start learning, have a look at the Reference and Security Links sections of the Shadow Web site. Then congratulate yourself for giving your organization a real chance to catch an intruder before he can do any damage.
- "Bastille" - http://www.bastille-linux.org/
- "Snort" - http://www.snort.org/
- "Shadow" - http://www.nswc.navy.mil/ISSEC/CID
- "ACID" - http://www.andrew.cmu.edu/user/rdanyliw/snort/snortacid.html
- "IDS Policy Manager" - http://www.activeworx.org/programs/idspm/index.htm
- "tcpdump" - http://www.tcpdump.org/
- "Shadow Web site" - http://www.nswc.navy.mil/
SHADOW Technical Analysis
ShadowCon '00 Presentations
ShadowCon '99 Presentations
Proposed 1999 DARPA Off-line Intrusion Detection Evaluation Plans
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : C++ Humor : ARE YOU A BBS ADDICT? : Object oriented programmers of all nations : C Humor : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor: Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : The Most Comprehensive Collection of Editor-related Humor : Microsoft plans to buy Catholic Church : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor : Best Russian Programmer Humor : Russian Musical Humor : The Perl Purity Test : Politically Incorrect Humor : GPL-related Humor : OFM Humor : IDS Humor : Real Programmers Humor : Scripting Humor : Web Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor :
The Last but not Least
|You can use PayPal to make a contribution, supporting hosting of this site with different providers to distribute and speed up access. Currently there are two functional mirrors: softpanorama.info (the fastest) and softpanorama.net.|
The statements, views and opinions presented on this web page are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.
Last updated: July 07, 2013