Documentation > Splunk > Admin Manual > Pre-process files with a specific extension

Admin Manual

 


Overview
Setting Up Data Inputs
Authentication
Changing the Way Splunk Processes Your Data
Creating and Using Configuration Bundles
Configuring Your Sources
Splunk-2-Splunk Management
Integration
Managing Your Filesystem
How-To's

Pre-process files with a specific extension

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Pre-process files with a specific extension

The Splunk Server can be configured to run a pre-processor scipt on files before it processes them internally. This is especially useful for files with binary formats that can't easily be parsed into searchable segments.


Preprocessing can only be used on files configured as batch inputs, rather than tail inputs or non-file sources.


The following example would run the imaginary script preprocessor.sh on all files whose name ends in .dat. It goes into the configuration file $SPLUNK_HOME/etc/bundles/local/props.conf. Note the four dots in a row. The first three are a wildcard for any path. The fourth is the start of the .dat file extension. 


[source::....dat] 
invalid_cause = needs_preprocess 
preprocessing_script = /bin/local/preprocessor.sh

This script should be in your path unless you specify an absolute path. Splunk's pre-processor scripts in $SPLUNK_HOME/bin - gunzipit, uncompressit, etc. - can be used as a template to create your own.

Retrieved from "http://docs.splunk.com/Documentation/Splunk/2.2.6/Admin/PreprocessFilesWithASpecificExtension"

This documentation applies to the following versions of Splunk: 2.1 , 2.2 , 2.2.1 , 2.2.3 , 2.2.6 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.