Tune timestamp recognition for better indexing performance
To speed up indexing, you can use the props.conf configuration file to adjust how far ahead into events you want the timestamp processor to look. You can even turn off the timestamp processor altogether.
If you use Splunk Cloud Platform and need to modify timestamp extraction, use a heavy forwarder and perform the configuration on the machines where the heavy forwarders run.
If you use Splunk Enterprise and need to modify timestamp extraction, perform the configuration on your indexer machines. If you are forwarding data, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run. If you use Splunk Cloud Platform and need to modify timestamp extraction, use a heavy forwarder and perform the configuration on the machines where the heavy forwarders run. For information on editing the props.conf configuration file for timestamps, see Configure timestamp recognition.
Adjust timestamp lookahead
Timestamp lookahead determines how many characters into an event the timestamp processor looks for a timestamp. Adjust how far the timestamp processor looks by adjusting the
The default number of characters that the timestamp processor looks into an event is 128. You can set the
MAX_TIMESTAMP_LOOKAHEAD setting to a lower value to speed up indexing. Do this if the timestamps always occur in the first part of the event.
This examples looks for timestamps in the first 20 characters of events coming from the source
[source::foo] MAX_TIMESTAMP_LOOKAHEAD = 20 ...
Disable timestamp processor
You can turn off the timestamp processor entirely to improve indexing performance. Turn off timestamp processing for events that match a specific host, source, or source type by configuring the
DATETIME_CONFIG setting to
DATETIME_CONFIG=NONE, Splunk software doesn't look at the text of the event for the timestamp. Instead, it uses the event time of receipt, or the time the event is received from its input. For file-based inputs such as
monitor, the timestamp comes from the modification time of the input file.
You can also increase indexing performance by setting the
DATETIME_CONFIG setting to
CURRENT, which assigns the current system time to each event at the time of indexing.
This example turns off timestamp extraction for events that come from the source
[source::foo] DATETIME_CONFIG = NONE ...
NONE disable timestamp identification, so the default event boundary detection,
BREAK_ONLY_BEFORE_DATE = true, might not work as you expect. When you use these settings, specify
SHOULD_LINEMERGE or the
MUST_BREAK_* settings to control event merging.
Specify time zones for timestamps
About indexed field extraction
This documentation applies to the following versions of Splunk® Enterprise: 6.5.7, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.1, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 8.0.0, 8.0.10, 8.0.2
Feedback submitted, thanks!