Strip syslog headers before processing

Strip syslog headers before processing

This example removes syslog headers from non-syslog events that have been passed through syslog to Splunk, such as log4j events from a log4j-to-syslog appender.

Splunk ships with a regex to do this for you.

etc/bundles/default/transforms.conf

# This will strip out date stamp, host, process with pid and just get the
# actual message
[syslog-header-stripper-ts-host-proc]
REGEX         = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s.*?:\s(.*)$
FORMAT        = $1
DEST_KEY      = _raw

Additional strippers found in this file include:

• syslog-header-stripper-ts-host-proc This will strip out date stamp, host, process with pid and just get the actual message
• syslog-header-stripper-ts-host This will strip the syslog header (date stamp and host) from a syslog event. This is especially useful in allowing Splunk to extract the correct hostname if you are using hostname chaining
• syslog-header-stripper-ts This will just strip the time stamp

etc/bundles/local/props.conf

This example turns on the built-in regex for remote syslog inputs.

[syslog]
TRANSFORMS= syslog-header-stripper-ts-host-proc

You can append helpful names onto the TRANSFORMS declaration, like this.

[syslog]
TRANSFORMS-strip-syslog= syslog-header-stripper-ts-host-proc

There are no special keywords. "TRANSFORMS-I-hate-them" would work just as well.

Example

If you have a central syslog server (syslog1.idkfa.kom) that is getting events from multiple servers, you can forward the events to a Splunk server and index the events based on the original host (doom1.idkfa.kom) and original timesamp (07:37:15). For this example the events are coming into Splunk via UDP port 514 and they look like this:

Mar 30 14:29:35 syslog1.idkfa.kom Mar 30 07:37:15 doom1.idkfa.kom sshd[7728]: Connection closed by ::ffff:192.168.1.101

Create this configuration stanza in props.conf:

[syslog]
TIME_PREFIX = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s[^\s]*\s
TRANSFORMS-strip-syslog= syslog-header-stripper-ts-host

• Was this topic useful?
• Post a comment