Perl-based Event Correlation

SEC has been primarily tested on Linux and Solaris, but since it is written in Perl and does not use any platform-dependent subroutines, it should work on most operating systems. The author has received reports about SEC working on FreeBSD, OpenBSD, HP-UX, AIX, Tru64 UNIX, Mac OS X, and Win2000 (with CygWin Perl).

Key features:

• input: regular files, named pipes, and standard input, and can thus be employed as an event correlator for any application that is able to write its output events to a file stream.
• rules, each rule specifying an event matching condition, an action list, and optionally a Boolean expression whose truth value decides whether the rule can be applied at a given moment. Regular expressions, Perl subroutines, etc. are used for defining event matching conditions. Event correlation rule types:
  • Single - match input event and execute an action list.
  • SingleWithScript - match input event and execute an action list, if an external script or program returns a certain exit value.
  • SingleWithSuppress - match input event and execute an action list, but ignore the following matching events for the next t seconds.
  • Pair - match input event, execute an action list, and ignore the following matching events until some other input event arrives. On the arrival of the second event execute another action list.
  • PairWithWindow - match input event and wait for t seconds for other input event to arrive. If that event is not observed within the given time window, execute an action list. If the event arrives on time, execute another action list.
  • SingleWithThreshold - count matching input events during t seconds and if a given threshold is exceeded, execute an action list and ignore the following matching events during the remaining time window. The window of t seconds is sliding.
  • SingleWith2Thresholds - count matching input events during t1 seconds and if a given threshold is exceeded, execute an action list. Then start the counting of matching events again and if their number per t2 seconds drops below the second threshold, execute another action list. Both event correlation windows are sliding.
  • Suppress - suppress matching input event (used to keep the event from being matched by later rules).
  • Calendar - execute an action list at specific times.

    Rules allow not only shell commands to be executed as actions, but they can also:

    • create and delete contexts that decide whether a particular rule can be applied at a given moment,
    • associate events with a context and report collected events at a later time (similar feature is supported by logsurfer),
    • generate new events that will be input for other rules,
    • reset correlation operations that have been started by other rules,
    • spawn external event, fault, or knowledge analysis modules,
    • etc.
    This makes it possible to combine several rules and form more complex event correlation schemes.
• output events can be produced by executing user-specified shell scripts or programs by writing messages to pipes or files, and by various other means. The applications SEC has been used or integrated with include HP OpenView NNM and Operations, CiscoWorks, BMC Patrol, Nagios, SNMPTT, Snort IDS, Prelude IDS, etc.

Old News ;-)

[Jan 30, 2006] "Simple Event Correlator for real-time security log monitoring" - a paper about SEC that was published in Hakin9 Magazine

Simple Event Correlation installation and configuration