Mad Irish . net

Writing OSSEC Custom Rules and Decoders

30 November -0001

OSSEC (http://www.ossec.net) is an open source host based intrusion detection system (HIDS). OSSEC can be used to monitor your local files and logs to check for intrusions, alert you of rootkit installation and do file integrity checking. OSSEC is a wonderful tool because it is highly customizable. By default OSSEC monitors many of the programs commonly installed on a machine, but it's real power comes from the ability of system administrators to customize OSSEC. By writing custom rules and decoders, you can allow OSSEC to parse through non-standard log files and generate alerts based on custom criteria. This allows OSSEC to monitor custom applications and provide intrusion detection services that might otherwise not be available, or would have to be developed on a per-application basis.

OSSEC rules are based on log file parsing. The log files that OSSEC monitors are specified in the /var/log/ossec/etc/ossec.conf file in the following format: 

  <!-- Files to monitor (localfiles) -->

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/messages</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/secure</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/maillog</location>
  </localfile>

  <localfile>
    <log_format>apache</log_format>
    <location>/var/log/httpd/error_log</location>
  </localfile>

  <localfile>
    <log_format>apache</log_format>
    <location>/var/log/httpd/access_log</location>
  </localfile>

Each file that is monitored depends on a "decoder" which is a regular expression used to parse up the pieces of the log file to extract fields such as the source IP, the time, and the actual log message. The decoders are stored in /var/ossec/etc/decoder.xml. The following is an extract of the SSH decoder portion of the decoder.xml logfile: 

<decoder name="sshd">
  <program_name>^sshd</program_name>
</decoder>

<decoder name="sshd-success">
  <parent>sshd</parent>
  <prematch>^Accepted</prematch>
  <regex offset="after_prematch">^ \S+ for (\S+) from (\S+) port </regex>
  <order>user, srcip</order>
  <fts>name, user, location</fts>
</decoder>

<decoder name="ssh-denied">
  <parent>sshd</parent>
  <prematch>^User \S+ from </prematch>
  <regex offset="after_parent">^User (\S+) from (\S+) </regex>
  <order>user, srcip</order>
</decoder>

You can see that the decoder.xml file is used to parse through the log file using regular expression pattern matching. This means that you can add additional files to the list of those which OSSEC is checking if you would like. You'll also note that the XML rules in decoder.xml are nested, so that you can use the <parent> tag to nest rules. A rule with a "parent" will only attempt matching if the parent rule matched successfully. Using the order and fts statements you can populate OSSEC's predefined variables with portions of the log file. The following variables are supported:

  • location
  • hostname
  • log_tag
  • srcip
  • dstip
  • srcport
  • dstport
  • protocol
  • action
  • user
  • dstuser
  • id
  • command
  • url
  • data

Supposing you have a log file produced by an application that isn't covered by the default decoders you could write your own decoder and parsing rules. Unfortunately OSSEC only supports logs in the formats syslog, snort-full, snort-fast, squid, iis, eventlog, mysql_log, postgresql_log, nmapg or apache. Therefore any custom logging you write must conform to one of these formats. Syslog is probably the easiest to use as it is designed to handle any one line log entry.

Let us suppose we have a custom PHP based application that resides in /var/www/html/custom. Our application will write Apache format logs to a file called 'alert.log' in the 'logs/' application subdirectory. This program has the following lines in example.php: 

<?php

$id = $_GET['id'];
$logfile = 'logs/alert.log';
if (! is_numeric($_GET['id'])) {
	$timestamp = date("Y-m-d H:m:s ");
	$ip = $_SERVER['REMOTE_ADDR'];
	$log = fopen($logfile, 'a');
	$message = $timestamp . $ip . ' PHP app Attempt at non-numeric input (possible attack) detected!' . "\n";
	fwrite($log, $message);
}

?>

This would write a log file to /var/www/html/custom/logs/alert.log in the format: 

2009-10-13 11:10:36 192.168.97.1 PHP app Attempt at non-numeric input (possible attack) detected!

Once we have this application log set up we need to adjust our OSSEC configuration so that it reads the new log file. We can add the following lines to our /var/ossec/etc/ossec.conf file to enable OSSEC to read this new log file: 

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/www/html/custom/logs/alert.log</location>
  </localfile>

Once OSSEC is monitoring this file (this will require us to restart OSSEC) we'll need an appropriate decoder. Writing a decoder for this format would be quite simple. It would appear as: 

<!-- Custom decoder for example -->
<decoder name="php-app">
  <prematch>^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d</prematch>
</decoder>

<decoder name="php-app-alert">
  <parent>php-app</parent>
  <regex offset="after_parent">^ (\d+.\d+.\d+.\d+) PHP app</regex>
  <order>srcip</order>
</decoder>

What we're doing here is telling OSSEC how to extract IP information from the log. All the strings in the regex portion of the new decoder can be assigned, in order, to options listed in the order tag. You can define each of OSSEC's possible variables and tell OSSEC how to identify them in the logs using the decoder.

Once we have our decoder we can write custom rules based on the log file. There are two ways to create custom rules for OSSEC. The first is to alter the ossec.conf configuration file and add a new rule file to the list. The second is to simply append your rules to the local-rules.xml rules file. Either one works, but the second makes upgrading to newer versions of OSSEC a little cleaner.

We'll add the following group to our local-rules.xml file, found in the rules directory under the OSSEC installation root: 

<group name="syslog,php-app,">
  <rule id="110000" level="0">
    <decoded_as>php-app</decoded_as>
    <description>PHP custom app group.</description>
  </rule>

  <rule id="110001" level="10">
    <if_sid>110000</if_sid>
    <srcip>127.0.0.1</srcip>
    <match>attack</match>
    <description>Possible attack from localhost?!?</description>
  </rule>
</group>

You'll notice that we have two rules. Because rules can be nested it is usually helpful to subdivide them into small, hierarchical pieces. In this case we have one rule that serves as a catch-all for our custom application alerts. After that we can write rules for any number of circumstances and have these rules only checked if the parent rule is matched. This rule will fire if an entry is written into the custom alert.log that contains the source IP of 127.0.0.1. The rule id is extremely important in this definition. OSSEC reserves rule id's above 100,000 for custom rules. It is useful to develop a schema for your new rules, for instance allocating each 1000 above 100,000 for a generic, catch-all rule and writing child rules in that space. This helps to avoid the hassle of having intermingled rule numbers and aids in long term maintenance.

To clarify the case above, there are two rules. The first rule will only fire if a log entry is "decoded_as" or matches the decoder for "php-app." If this decoder is used then rule 110,000 will be triggered. The second rule is only checked if rule 110,000 is triggered as specified in the if_sid tag. This rule will only be triggered if the source ip, specified in the srcip tag, is equal to '127.0.0.1.' If this is the case then the rule will do a string match for the word "attack" in the log entry. If this match is successful the rule will trigger at level 10 as specified in the rule tag. This will cause an OSSEC alert to be logged with the associated description. OSSEC by default also attempts to e-mail alerts with level 7 or higher to recipients specified in the ossec.conf file. As you can see, with the addition of the decoder and these rules we've allowed OSSEC to read our custom format logfile.

While this example may seem straightforward writing your own decoders and rules can be maddening. Because OSSEC will not dynamically load the XML files defining your decoders, rules, or files to watch, you must restart the program to propagate changes. This can be a real hassle when you're debugging new XML rules or decoders. To alleviate the problem of constantly restarting the server you can use the program ossec-logtest found in the bin directory of the OSSEC installation root. This program allows you to paste, or type, one line of a log file into the input then traces the decoders and rules that the line matches like so: 

# bin/ossec-logtest 
2009/10/13 13:30:25 ossec-testrule: INFO: Started (pid: 14330).
ossec-testrule: Type one log per line.

2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!


**Phase 1: Completed pre-decoding.
       full event: '2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!'
       hostname: 'webdev'
       program_name: '(null)'
       log: '2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!'

**Phase 2: Completed decoding.
       decoder: 'php-app'
       srcip: '127.0.0.1'

**Phase 3: Completed filtering (rules).
       Rule id: '110001'
       Level: '10'
       Description: 'Possible attack from localhost?!?'
**Alert to be generated.

Note that this program will not reload changes, but you can quit ossec-logtest, make changes to any of the XML files then restart it to test your changes. Using ossec-logtest is invaluable when trying to create new rules as it saves you the hassle of restarting the server and the hassle of actually triggering events for which you want to generate alerts.

Conclusion

Writing custom OSSEC decoders and rules allows you to extend the power of OSSEC's host based intrusion detection system to custom and non-standard applications running on your OSSEC monitored machines. By leveraging the power of OSSEC to do this sort of log analysis and alerting you can avoid the hassle of building intrusion detection into your existing applications. Although applications will require some sort of logging in order to enable this behavior it is much simpler to build or enable logging for applications than it is to attempt to devise intrusion detection systems that run natively in applications. By customizing your alerts you can change OSSEC's behavior to meet the unique needs of your situation and extend the power of your OSSEC installation.

Tags