How Splunk Handles Data
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
- Splunk is handling multiple events as one. Splunk is splitting multiple line events in the wrong place. How do I fix that?
- Splunk is not recognizing my timestamps correctly. How do I fix that?
- I want to add custom fields such as user:: to the default fields like source::. How do I do that?
- I have some sensitive data. Can I garble it before it gets indexed?
- I want to search for messages in my email logs based on both sender and recipient, but these are recorded in different events with a common message id. Can Splunk handle that?
- Does Splunk read milliseconds?
How Splunk Handles Data
Splunk is handling multiple events as one. Splunk is splitting multiple line events in the wrong place. How do I fix that?
You can override Splunk's default multiline event handling rules by editing properties set by source or sourcetype. Complete instructions can be found in the Admin Manual. You can see examples in the example files in $SPLUNK_HOME/splunk/etc/bundles/.
Splunk is not recognizing my timestamps correctly. How do I fix that?
You can train Splunk to recognize timestamps better. Run $SPLUNK_HOME/bin/splunk train dates to teach Splunk the dates to extract from your datasources.
I want to add custom fields such as user:: to the default fields like source::. How do I do that?
You can specify additional fields to be indexed or extracted at search time in properties configuration files. Instructions can be found in the Admin Manual. You can see examples in the example files in $SPLUNK_HOME/etc/bundles/props.conf.example and $SPLUNK_HOME/etc/bundles/transforms.conf.example.
I have some sensitive data. Can I garble it before it gets indexed?
Yes. There is an anonymizer you can use to maintain confidentiality. Please see the section of our Admin Manual on Anonymizing your Data Samples.
I want to search for messages in my email logs based on both sender and recipient, but these are recorded in different events with a common message id. Can Splunk handle that?
Yes. You will have to configure Splunk to recognize the common field and tell it to create meta-events to summarize all events with that field in common. You can see examples in the example files in your $SPLUNK_HOME/etc/bundles directory and in the Admin Manual.
Does Splunk read milliseconds?
We are not storing timestamps in intervals smaller than a second. You need to extract the ms time from the event and then write it to a custom search (indexed) field. Documentation on how to write a custom search field can be found here.
If you did this and called it ms, you would then pipe the search to the sort command and use the _time field and your custom field.
foo | sort _time, ms desc
_time is UTC time in seconds.
This documentation applies to the following versions of Splunk: 3.0 , 3.0.1 , 3.0.2 , 3.1 , 3.1.1 , 3.1.2 , 3.1.3 , 3.1.4 View the Article History for its revisions.