Documentation > Splunk > Admin Manual > Tune timestamp extraction for better indexing performance

Admin Manual

 


How Splunk Works
Getting Started
Data Inputs
Data Distribution
Indexing
Timestamps
Fields
Hosts
Source Types
Event Types
Transaction Types
Search
Distributed Search
Security
Data Management
Deployment Server
Performance Tuning
Configuration Files
Applications
Reference
Troubleshooting

Tune timestamp extraction for better indexing performance

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Tune timestamp extraction for better indexing performance

Tune Splunk's timestamp extraction by editing props.conf. Adjust how far Splunk's timestamp processor looks into events, or turn off the timestamp processor to make indexing faster.


Note: Use $SPLUNK_HOME/etc/bundles/README/props.conf.example as an example, or create your own props.conf. Make any configuration changes to a copy of props.conf in $SPLUNK_HOME/etc/bundles/local/, or your own custom bundle directory. For more information on configuration files in general, see how configuration files work.


Adjust timestamp lookahead

Timestamp lookahead determines how far (how many characters) into an event the timestamp processor looks for a timestamp. Adjust how far the timestamp processor looks by setting a value (the number of characters) for the MAX_TIMESTAMP_LOOKAHEAD = key in any timestamp stanza.


Note: You can set MAX_TIMESTAMP_LOOKAHEAD = to different values for each timestamp stanza.


The default number of characters that the timestamp processor looks into an event is 150. Set MAX_TIMESTAMP_LOOKAHEAD = to a lower value to speed up how fast events are indexed. You should do this if your timestamps occur in the first part of your event.


If your events are indexed in real time, increase Splunk's overall indexing performance by turning off timestamp lookahead (set MAX_TIMESTAMP_LOOKAHEAD = 0). This causes Splunk to not look into event's for a timestamp, and sets an event's timestamp to be its indexing time (using current system time).


Example:


This example tells the timestamp processor to look 20 characters into events from source foo. 


[source::foo]
MAX_TIMESTAMP_LOOKAHEAD = 20
...

Turn off the timestamp processor

Turn off the timestamp processor entirely to significantly improve indexing performance. Turn off timestamp processing for events matching a host, source, sourcetype specified by a timestamp stanza by adding a DATETIME_CONFIG = key to a stanza and setting the value to NONE. When timestamp processing is off, Splunk won't look for timestamps to extract from event data. Splunk will instead set an event's timestamp to be its indexing time (using current system time).


Example:


This example turns off timestamp extraction for events that come from the source foo. 


[source::foo]
DATETIME_CONFIG = NONE
...
Retrieved from "http://docs.splunk.com/Documentation/Splunk/3.2.2/Admin/TuneTimestampExtractionForBetterIndexingPerformance"

This documentation applies to the following versions of Splunk: 3.2.2 , 3.2.3 , 3.2.4 , 3.2.5 , 3.2.6 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!