Documentation > Splunk > Admin Manual > Tune timestamp extraction for better indexing performance

Admin Manual

 


Welcome
What to do first
Start Splunk
Meet Splunk Web and Splunk Apps
Before you configure
Add data and configure inputs
Indexing and event processing
Timestamps
Add and manage users
Manage indexes
Define alerts
Set up backups and retention policies
Configure data security
Set up forwarding and receiving
Deploy to other Splunk instances
Set up distributed search
Manage search jobs
Use Splunk's command line interface (CLI)
Configuration file reference
Troubleshooting

Tune timestamp extraction for better indexing performance

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Tune timestamp extraction for better indexing performance

Tune Splunk's timestamp extraction by editing props.conf. Adjust how far Splunk's timestamp processor looks into events, or turn off the timestamp processor to make indexing faster.

Note: Use $SPLUNK_HOME/etc/system/README/props.conf.example as an example, or create your own props.conf. Make any configuration changes to a copy of props.conf in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work.

Adjust timestamp lookahead

Timestamp lookahead determines how far (how many characters) into an event the timestamp processor looks for a timestamp. Adjust how far the timestamp processor looks by setting a value (the number of characters) for the MAX_TIMESTAMP_LOOKAHEAD = key in any timestamp stanza.

Note: You can set MAX_TIMESTAMP_LOOKAHEAD = to different values for each timestamp stanza.

The default number of characters that the timestamp processor looks into an event is 150. Set MAX_TIMESTAMP_LOOKAHEAD = to a lower value to speed up how fast events are indexed. You should do this if your timestamps occur in the first part of your event.

If your events are indexed in real time, increase Splunk's overall indexing performance by turning off timestamp recognition. Set DATETIME_CONFIG = CURRENT to cause Splunk to not look into events for their timestamps and instead assign the current system time (at the time of indexing) to each event.

Example:

This example tells the timestamp processor to look 20 characters into events from source foo. 

[source::foo]
MAX_TIMESTAMP_LOOKAHEAD = 20
...


Disable timestamp determination

Turn off the timestamp processor entirely to significantly improve indexing performance. Turn off timestamp processing for events matching a host, source, sourcetype specified by a timestamp stanza by adding a DATETIME_CONFIG = key to a stanza and setting the value to NONE. When timestamp processing is off, Splunk does not look at the text of the event for the timestamp--it instead uses the event's "time of receipt"; in other words, the time the event is received via its input.

For file-based inputs (such as monitor) this means that Splunk derives the event timestamp from the modification time of the input file.

Example:

This example turns off timestamp extraction for events that come from the source foo. 

[source::foo]
DATETIME_CONFIG = NONE
...
Retrieved from "http://docs.splunk.com/Documentation/Splunk/4.0/Admin/TuneTimestampExtractionForBetterIndexingPerformance"

This documentation applies to the following versions of Splunk: 4.0 , 4.0.1 , 4.0.2 , 4.0.3 , 4.0.4 , 4.0.5 , 4.0.6 , 4.0.7 , 4.0.8 , 4.0.9 , 4.0.10 , 4.0.11 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.