WFLOGS

wflogs − firewall log analyser of the WallFire project.

wflogs [options] [logfile]

wflogs is a firewall log analyser. It can be used to produce a log summary report in plain text, HTML and XML, or even to translate a log file into another firewall log format, for example. Logs can be filtered, summarized, sorted, and obfuscated (in that order), using the following options.
By default, output is not sorted, and may be summarized if the output module has a ‘summary’ option and if this option is set to ‘yes’ (even by default value).
You have to specify a module name that will handle the input (parsing) and another for the output (exportation). See MODULES sections below.
With no logfile, wflogs read /var/log/messages. When logfile is ‘-’, it reads standard input.

-c | --config file

wflogs will use given configuration file. If not specified, wflogs will not use any configuration file and will only use command line options.

-f | --filter expression

Print log entries that match the boolean expression. This expression looks very much like a Perl condition, which must be passed as a single, quoted argument. If no expression is given, all log entries will be dumped. Otherwise, only entries for which expression is ‘true’ will be dumped. See the FILTER EXPRESSION section below.

-i | --input-format format[,format2,...]

Specify the input parsing modules. Wflogs will use the corresponding modules (if available) to parse the logs. If you want to parse a log file with multiple formats mixed (typically a remote syslog file), you can specify several format module names separated by commas, one being probed after another. Use special name ‘all’ to try every available format. If you omit the ‘-i’ option, wflogs will try to guess the local firewalling tool at runtime, and use the corresponding module. Use format ‘help’ to know which modules are available (currently, ‘netfilter’, ‘ipchains’, ‘ipfilter’, ‘cisco_pix’, ‘cisco_ios’, and ‘snort’), and which is the default (guessed) module. See INPUT MODULES section below.

-I, --interactive

Interactive mode. The program will not terminate, but enter a little interactive shell.
This option can be used in conjonction with real-time mode (-R option). While in non-interactive real-time mode (-R only), signal USR1 enables to fall back into interactive mode.

-o | --output-type type [ output module options ]

Specify the output module type. Wflogs will use the corresponding module (if available) to export the input logs to the corresponding target. Use type ‘help’ to know which modules are available (currently, ‘text’, ‘html’, ‘xml’, ‘human’, ‘netfilter’, ‘ipchains’, and ‘ipfilter’). Default mode is ‘text’. See OUTPUT MODULES section below.
Output module configuration can be achieved via the command line. You can specify long options (with a ‘--’ prefix). Three types are supported: boolean (yes or no), integer, and string. A special option ‘--options’ displays the available options of the module, with type, help message, and default value. For example, wflogs -o html --options shows the HTML output module configuration.

-O | --obfuscate [criterias]

This option obfuscates some logging fields according to given criterias, separated by commas. These can be ‘date’, ‘hostname’, ‘ipaddr’, or ‘macaddr’ (or ‘all’ for everything). Default (if no criteria is given) is ‘all’. If ipaddr is specified, output module options ‘resolve’ and ‘whois_lookup’ (if available) are set to no. If macaddr is specified, output module option ‘mac_vendor’ (if available) is set to no.
Date order is conserved, hostnames are replaced by "hostx" (where x is a growing number), ipaddr belong to 0.0.0.0/8 network, macaddr are of the form 0:0:0:0:0:1, 0:0:0:0:0:2, etc. Note that for all obfuscated fields, each original value is associated with a new unique one (unicity is preserved).

-P | --proceed

If real-time (-R) or interactive (-I) modes are set, first process log entries in the input logfile before entering in these modes, as these entries won’t be parsed by default in these modes.

-R | --realtime

‘‘Real-time’’ mode: logs are monitored in real-time. Wflogs will wait for new log entries. Entries already present in the input logfile will not be processed as usual, unless you specify -P option.
This option can be used in conjonction with interactive mode (-I option). While in non-interactive real-time mode (-R only), signal USR1 enables to fall back into interactive mode.

-s | --sort[=criteria_list]

Set output lines sort order according to the multilevel sort specified by the sequence of keys key1,key2,... Syntax is --sort=[+|-]key1[,[+|-]key2[,...]]. Choose a key from the SORT KEYS section. ‘-’ reverses direction only on the key it precedes. The ‘+’ is really optional since default direction is increasing numerical or lexicographic order. For example wflogs --sort=dport,-time sorts according to destination port number, then reverse time (for a given port number). If one of the keys is ‘none’, the output is not sorted. Use key ‘help’ to show available keys. If no sort criteria is given, output is sorted by with ‘-count,time,dipaddr,protocol,dport’.

--strict-parsing type

Set the parsing policy. Available types are: ‘loose’ (even if there are garbage in the input file or incorrect log lines, parse as much as possible and issue no warning at all), ‘nowarning’ (in this case, issue no warnings, ignore non-log lines but do not store incoherent entries), ‘warning’ (issue warnings on stderr, ignore non-log lines but do not store bizarre entries), and ‘error’ (stop parsing if line is not a log entry, or if entry is bizarre). Default type is ‘warning’.

-v | --verbose [level]

Set verbosity level. If level is omitted, default value is 1.

-V | --version

Display current version.

-h | --help

Show help message on stdout.

wflogs can use extended input modules, each one parsing a specific firewall log format. See option -i.

netfilter

This module parse the netfilter log format.

ipchains

This module parse the ipchains log format.

ipfilter

This module parse the ipfilter log format.

cisco_pix

This module parse the cisco PIX and cisco FWSM log format.

cisco_ios

This module parse the cisco IOS log format.

snort

This module parse the snort IDS ACLs log format.

wflogs can use extended output modules, which enable to export the input logs to a particular format. So it can be used to rewrite the input into another firewall log format or generate a report, for example. See option -o. Summary mode depends on the module, and is configurable through the ‘summary’ module option.

text

This module produces a summary in text mode. Please note that this text output is not intended to be parsed. Use XML output module instead.

html

This module produces a summary output in HTML format.

xml

This module produces a summary in XML format (see wflogs DTD).

human

This module produces a summary in text format, in a human readable form. Newcomers may like it. ;-)

netfilter

This module exports input logs to netfilter log syntax.

ipchains

This module exports input logs to ipchains log syntax.

ipfilter

This module exports input logs to ipfilter log syntax.

count

sort by count (number of original log entries)

time

sort by log entry date (if count != 1, the date of the

first original log line)

timeend

sort by log entry end date (if count != 1, the date of the

last original log line)

input_iface

sort by input interface name

output_iface

sort by output interface name

sipaddr

sort by source IP address

dipaddr

sort by destination IP address

smacaddr

sort by source MAC address

dmacaddr

sort by destination MAC address

protocol

sort by protocol number

sport

sort by source port number (if available)

dport

sort by destination port number (if available)

tcpflags

sort by TCP flags

hostname

sort by hostname

chainlabel

sort by chain label

branchname

sort by branch name

datalen

sort by data length

format

sort by firewalling tool format

none

do not sort

This filtering expression looks very much like a Perl condition.

Variables are prefixed with ‘$’. Pre-defined variables are:

$format (string)

firewalling tool format

$count (integer)

number of original log entries

$start_time ([string] or integer)

log entry date (if count != 1, the date of the first original log line), in date format ([string], see below), or in seconds since the Epoch

$end_time ([string] or integer)

log entry end date (if count != 1, the date of the last original log line), in date format ([string], see below), or in seconds since the Epoch

$hostname (string)

name of the host which logged the packet

$chainlabel (string)

chain label

$branchname (string)

branch name

$input_iface (string)

input interface name

$output_iface (string)

output interface name

$protocol (integer)

protocol number (or name used in /etc/protocols)

$datalen (integer)

data length

$sipaddr (IP network)

source IP address, or source IP network

$sport (integer)

source port number (or name used in /etc/services) if protocol is UDP or TCP, and ICMP type number or name if protocol is ICMP (this may change in the future)

$smacaddr (MAC address)

source MAC address

$dipaddr (IP network)

destination IP address, or destination IP network

$dport (integer)

destination port number (or name used in /etc/services) if protocol is UDP or TCP, and ICMP code number or name if protocol is ICMP (this may change in the future)

$dmacaddr (MAC address)

destination MAC address

$tcpflags (integer)

TCP flags if protocol is TCP (flags can be a combination of SYN|ACK|RST|FIN|PSH|URG|ECE|CWR)

For integer and boolean values, the following operators can be used: ||, &&, ==, !=, <, >, <=, >=, &, |, ^, +, -.

String variables can be compared for strict equality with == and != operators, but also matched with an extended regular expression with =~ operator. Strings are quoted with " (like "foo"), and regexps with / (like /(foo|bar)/). Note that regexp matches only a subset of the string. You have to surround the regexp with ^ and $ if you want to match the whole string (that may change in the future). Like in Perl, you may add an optional i modifier after final /, to do case-insensitive pattern matching.

Date format is one that is accepted by the getdate C function. It must be enclosed in brackets [] and will be converted to an integer value which stands for the number of seconds since the epoch (01 Jan 1970 UTC 00:00). See DATE FORMAT section.

IP network can be an IP address, or an IP network (a.b.c.d/n.o.p.q or a.b.c.d/bitmask, or even things like a.b.*.* for a /16 mask, for example).

MAC addresses are of the form aa:bb:cc:dd:ee:ff. They can only be compared for strict equality (== and != operators).

The string may contain many flavors of items: calendar date items, time of the day items, time zone items, day of the week item, relative items, or pure numbers. As expression can be quite complex, if you have doubt about the dates you specified, activate global verbose mode to show filter expression on stderr using absolute dates.

Calendar date

can be "1974-08-31", "74-8-31", "74-08-31", "8/31/74", "31 August 1974", "31 Aug 1974", "Aug 31, 1974", "31-aug-74", "31aug74". The year can be omitted (current year is then used).

Time of day

can be "02:50:00", "02:50", "2:50am".

Day of week

can be "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday" or "Saturday", but can be abbreviated to their first three letters. A number may precede a day of the week item to move forward supplementary weeks. It is best used in expression like ‘third monday’. In this context, ‘last DAY’ or ‘next DAY’ is also acceptable; they move one week before or after the day that DAY by itself would represent.

Relative items

adjust a date (or the current date if none) forward or backward. It can be "1 year", "1 year ago", "3 years", "2 days", for example. You can also use "month", "week", "day", "hour", "minute" ("min"), and "second" ("sec"), or "now" ("today"), "yesterday", and "tomorrow". The string ‘this’ also has the meaning of a zero-valued time displacement, but is preferred in date strings like ‘this thursday’.

Pure decimal number

precise intepretation depends on the context in the date string. If the decimal number is of the form YYYYMMDD and no other calendar date item appears before it in the date string, then YYYY is read as the year, MM as the month number and DD as the day of the month, for the specified calendar date. If the decimal number is of the form HHMM and no other time of day item appears before it in the date string, then HH is read as the hour of the day and MM as the minute of the hour, for the specified time of the day. MM can also be omitted.

wflogs -i netfilter -o html netfilter.log > logs.html
converts the given netfilter log file into a HTML report.

wflogs --sort=protocol,-time -i netfilter -o text netfilter.log > logs.txt
converts the given netfilter log file into a sorted (by protocol number, then reverse time) text report.

wflogs -f ’$start_time >= [this 3 days ago] && $start_time < [this 2 days ago] && $chainlabel =~ /(DROP|REJECT)/ && $sipaddr == 10.0.0.0/8 && $protocol == tcp && ($dport == ssh || $dport == telnet) && ($tcpflags & SYN)’ -i netfilter -o text --summary=no
shows log entries (without summary) which match the given expression (refused connection attempts that occured 3 days ago to ssh and telnet ports coming from internal network 10.0.0.0/8).

wflogs -i netfilter --resolve=0 --whois=0 netfilter.log
converts the given netfilter log file into a text report (default mode), disabling IP address reverse lookups and whois lookups.

wflogs -i netfilter -o xml netfilter.log > logs.xml
exports netfilter logs in XML.

wflogs -i ipchains -o netfilter ipchains.log > netfilter.log
converts ipchains logs into netfilter log format. So you may process them with your favorite netfilter log analyser, for example (even if the latter may not be better than wflogs itself. ;-)).

wflogs -i ipfilter -o human --datalen=yes ipfilter.log
produces a report about ipfilter logfile in natural language on stdout, displaying packet length (datalen option) which is not showed by default.

wfconvert(8), regex(7).

Bugs? What’s this? ;-) Contributions are welcome, please see http://wallfire.org/.

wflogs has been written by Herve Eychenne. See http://wallfire.org/.

This man page has been initiated by Gregoire Hubert <greg@coolkeums.org>, and written by Herve Eychenne.