Strip syslog headers before processing

Strip syslog headers before processing

Remove syslog headers from non-syslog events that have been passed through syslog to Splunk, such as log4j events from a log4j-to-syslog appender. Splunk ships with a regex to do this for you in $SPLUNK_HOME/etc/bundles/default/transforms.conf. Overwrite or change any of the default attributes and values by creating a transforms.conf in $SPLUNK_HOME/etc/bundles/local/ or your own custom bundle directory. For more information on configuration files in general, see how configuration files work.

Configuration

transforms.conf

In $SPLUNK_HOME/etc/bundles/default/transforms.conf:

# This will strip out date stamp, host, process with pid and just get the
# actual message
[syslog-header-stripper-ts-host-proc]
REGEX         = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s.*?:\s(.*)$
FORMAT        = $1
DEST_KEY      = _raw

Additional strippers found in this file include:

• syslog-header-stripper-ts-host-proc This will strip out date stamp, host, process with pid and just get the actual message
• syslog-header-stripper-ts-host This will strip the syslog header (date stamp and host) from a syslog event. This is especially useful in allowing Splunk to extract the correct hostname if you are using hostname chaining
• syslog-header-stripper-ts This will just strip the time stamp

props.conf

In $SPLUNK_HOME/etc/bundles/local/props.conf:

[syslog]
TRANSFORMS= syslog-header-stripper-ts-host-proc

This example turns on the built-in regex for remote syslog inputs.

[syslog]
TRANSFORMS-strip-syslog= syslog-header-stripper-ts-host-proc

Add a name onto the TRANSFORMS declarations. There are no special keywords. TRANSFORMS-the-cake-is-a-lie works just as well.

Example

If you have a central syslog server (syslog1.idkfa.kom) receiving events from multiple servers, you can forward the events to a Splunk Server and index them based on the original host (doom1.idkfa.kom) and original timestamp (07:37:15). For this example the events come to Splunk via UDP port 514 and look like this:

Mar 30 14:29:35 syslog1.idkfa.kom Mar 30 07:37:15 doom1.idkfa.kom sshd[7728]: Connection closed by ::ffff:192.168.1.101

Create this configuration stanza in props.conf:

[syslog]
TIME_PREFIX = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s[^\s]*\s
TRANSFORMS-strip-syslog= syslog-header-stripper-ts-host

• Was this topic useful?
• Post a comment