Documentation > Splunk > Admin Manual > Modify or delete segments within events

Admin Manual

 


Overview
Setting Up Data Inputs
Authentication
Changing the Way Splunk Processes Your Data
Creating and Using Configuration Bundles
Configuring Your Sources
Splunk-2-Splunk Management
Integration
Managing Your Filesystem
How-To's

Modify or delete segments within events

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Modify or delete segments within events

This example replaces every numeric pattern that looks like a US Social Security number - 253-45-3239 . It keeps the parts of the event before and after a match, but replaces the match with the text "xxx-xx-xxxx".


etc/bundles/local/regexes.conf

First, create a regular expression that matches numeric patters that look like US Social Security numbers - 423-78-8435 - and use the FORMAT attribute to place the parts of each event before and after an SSN around the string "xxx-xx-xxxx" instead of the SSN itself. 


[ssn_remover]
DEST_KEY = _raw
REGEX = ^(.*)\d{3}-\d{2}-\d{4}(.*)$
FORMAT = $1xxx-xx-xxxx$2

etc/bundles/local/props.conf

Then add an entry to map the regex function to a sourcetype. 


[my_custom_sourcetype]
REGEXES_remove-ssn = ssn_remover

Real-world example: Strip chained timestamp and hostname

How would a user construct a filter on the following event to strip off the leading "Oct 23 16:37:33 xxxxx.yyy.com"


Oct 23 16:37:33 xxxxx.yyy.com Oct 23 16:37:33 xxxxx.yyy.com MSWinEventLog 1 Security 39226 Mon Oct 23 16:37:25 2006 538 Security ZZZZZZZZ1$ User Success Audit xxxxx Logon/Logoff User Logoff: User Name: ZZZZZZZZ1$ Domain: yyy Logon ID: (0x0,0x475932G) Logon Type: 3 39211

The events come in via udp on port 6161 and the sourcetype is windows_snare_syslog...


Splunk ships with strippers in its default/regexes.conf. In local/props.conf find the sourcetype that you have associated with that log file and add syslog-header-stripper-ts to the REGEXES statement. If there is not a stanza already in props.conf the stanza would like: 


	[windows_snare_syslog]
	MAX_TIMESTAMP_LOOKAHEAD = 32
	REGEXES = syslog-header-stripper-ts,syslog-host,eventid

Real-world example

Customer wants Splunk to ignore the following event which gets printed repeatedly to one of his tailed files:


Nov 2 12:36:06 aikcvmwpr04lan CROND[26802]: (root) CMD (/usr/bin/vmkusage > /dev/null 2>&1)


In order to filter out the specific event you will need to do 3 things:


  1. Create a null queue for those events to go into
  2. Create an entry in props.conf for that source (or sourcetype) that will call the appropriate regex
  3. Create a regex in regex.conf that when matches sends the event to the null queue

Create a null queue:


In $SPLUNK_HOME/etc/myinstall/splunkd.xml add the following stanza after the last processor 


	<pipeline name="nullpipeline">
		<processor name="matt_null_processor" plugin="queueinputprocessor">
			<config>
				<queueName>nullQueue</queueName>
			</config>
		</processor>
	</pipeline>

Create an entry in props.conf:


In $SPLUNK_HOME/etc/bundles/local/props.conf add the following stanza: 


	[SOURCE::/var/log/splunk/syslog-ng/vmware.log]
	REGEXES = nullQueueRegex

Create an entry in regexes.conf:


In $SPLUNK_HOME/etc/bundles/local/regexes.conf add the following stanza: 


    [nullQueueRegex]
	REGEX = CROND
    DEST_KEY = queue
    FORMAT = nullQueue
Retrieved from "http://docs.splunk.com/Documentation/Splunk/2.2.1/Admin/ModifyOrDeleteSegmentsWithinEvents"

This documentation applies to the following versions of Splunk: 2.1 , 2.2 , 2.2.1 , 2.2.3 , 2.2.6 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!