Summarize related events as a meta event

Summarize related events as a meta event

Meta events are artificial events created by Splunk. They combine multiple events that match a common segment value, such as an IP address or username, into a single event. This is great for sendmail logs, where a single email transaction can be split across dozens of events. Splunk automatially creates meta events out of events whose source type is sendmail so that you can search for, say, a sender and a recipient, and find all events pertaining to the message transfer between them - without needing to deduce the unique message-ID value yourself and then go Splunking for all events with it. They're already collected in one meta event in Splunk's metaevents index. You can find them serendiptously, by searching for two usernames that happen to have a common meta event. Or you can add index::metaevents to Splunk deliberately for meta events.

The example below looks for IP addresses and creates a meta event for each IP address. The meta event contains all event that contain that IP address.

etc/bundles/local/regexes.conf

First, configure a regular expression to match IP addresses and store them in the special destination key cluster , which is where meta events go. We won't write meta data value, since it's already in the event as plain text.

[access-ip]
DEST_KEY = cluster
REGEX = (\d+\.\d+\.\d+\d+)
FORMAT = ip::$1

etc/bundles/local/props.conf

Then, add an entry to map the regular expression to specific source types.

[access_common]
REGEXES-cluster = access-ip

Transitive Meta Events

Events can also be linked transitively - if events A and B have a common value, and events B and C have a different common value, then all three can be part of the same meta event.

In the example below, events are being generated by a Sendmail Server and then forwarded to an MS Exchange server. The Exchange events all have the message ID in common, while the sendmail events have the QID in common. The only thing that joins the meta event together is a single event in the sendmail log that has both the message ID and QID in it.

etc/bundles/local/regexes.conf

#From the exchange events we are going to extract the message ID which has the form: 200703261746.c2YLgB00001000@shemika.paulette.com
[exchange_msgID]
DEST_KEY = cluster
REGEX = (\w+\.\w+@[^ ]+\.com)
FORMAT = messageID::$1
#From the sendmail log we are going to extract the QID which has the form: c2YLgB00001000 as well as the message ID
[sendmail_QID_msgID]
DEST_KEY = cluster
REGEX = sendmail\[\d+\]:\s+(\w+):.*\<(\w+\.\w+@[^ ]+\.com)\>
FORMAT = qid::$1 messageID::$2

etc/bundles/local/props.conf

#Call the regexes from regexes.conf 
[sendmail_syslog]
REGEXES-cluster = sendmail_QID_msgID
[exchange_custom]
REGEXES-cluster = exchange_msgID 

Sample MS Exchange event:

1006-3-23 17:46:25 GMT 129.24.20.020 shemika.paulette.com OLCP05.pr.paulette.com GDQEXMPX00 022.00.15.002 maritza@paulette.com 1000 200703261746.c2YLgB00001000@shemika.paulette.com 0 0 0060 5 1006-3-23 17:46:24 GMT 0 Version: 6.0.3440.0520 - Material Request - DON Group Org 000 : PMKOyige : 1 : Kai Frank Barb.Workflow@paulette.com -

Sample Sendmail events:

Mar 26 10:46:24 shemika sendmail[13005]: c2YLgB00001000: from=<Barb.Workflow@paulette.com>, size=454, class=0, nrcpts=1, msgid=<200703261746.c2YLgB00001000@shemika.paulette.com>, proto=SMTP, daemon=MTA, relay=oxgzl21.paulette.com [022.00.00.22]

Mar 26 10:46:24 shemika sendmail[13005]: c2YLgB00001000: list: /mailing-lists/barb.workflow

Mar 26 10:46:24 shemika sendmail[13005]: c2YLgB00001000: to=splunk@paulette,donte@paulette, delay=00:00:02, xdelay=00:00:00, mailer=daren, pri=130050, relay=gdqexmpx00.paulette.com. [022.00.15.002], dsn=2.0.0, stat=Sent ( <200703261746.c2YLgB00001000@shemika.paulette.com> Queued mail for delivery)

• Was this topic useful?
• Post a comment