Configure timestamp assignment for events with multiple timestamps
If an event contains more than one timestamp, you can specify which timestamp the event is to use for indexing. Configuring the timestamp is especially useful when you are indexing events that contain syslog host-chaining data.
Configure positional timestamp extraction by editing the props.conf configuration file. While there is limited ability to configure timestamp extraction in Splunk Web, you can achieve the best results by using configuration files on a heavy forwarder. See props.conf for more information on this configuration file.
- Splunk Cloud Platform
- On a Splunk Cloud Platform instance, you can configure timestamps on a heavy forwarder after you configured that forwarder to send data to the Splunk Cloud Platform instance. If you have access to the Edge Processor solution, you can also configure timestamps using Edge Processors by writing a custom regular expression in SPL2 to match and extract timestamps. See Extract timestamps from event data using an Edge Processor in the Use Edge Processors manual. For general information on editing the props.conf file for timestamps, see Configure timestamp recognition.
- Splunk Enterprise
- To modify timestamp extraction, perform the configuration on your indexer machines or, if you are forwarding data, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run.
- Prerequisites
- Only users with file system access, such as system administrators, can <describe the customization> using configuration files.
- Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.
- You can have configuration files with the same name in your default, local, and app directories. Read Where you can place (or find) your modified configuration files in the Splunk Enterprise Admin Manual.
Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make changes to the files in the local directory.
- Steps
- Open or create a local
<props.conf_file>
file at $SPLUNK_HOME/etc/system/local. - Add
TIME_PREFIX
andMAX_TIMESTAMP_LOOKAHEAD
settings to a stanza in the props.conf file.
By setting a regular expression forTIME_PREFIX
, you specify the pattern of characters that indicates the point to start looking for the timestamp. - Set a value for the
MAX_TIMESTAMP_LOOKAHEAD
setting to specify how far into an event past theTIME_PREFIX
location to look for the timestamp.
By constraining the amount of time to look ahead, you can improve both the accuracy and performance in determining and extracting the timestamp.
- When you set
TIME_PREFIX
setting, the Splunk platform scans the event text for a match to its regular expression before it tries to extract a timestamp. The timestamp algorithm looks for a timestamp in the text following the end of the first regular expression match. For example, if theTIME_PREFIX
setting is set toabc123
, only the text following the first occurrence ofabc123
is used for timestamp extraction.
- The
TIME_PREFIX
setting also sets the start point for theMAX_TIMESTAMP_LOOKAHEAD
setting. The lookahead starts after it finds the matched portion of text in theTIME_PREFIX
regular expression. For example, ifTIME_PREFIX
matches text through the first 11 characters of the event and the timestamp you want to extract is always within the next 30 characters, you can setMAX_TIMESTAMP_LOOKAHEAD=30
. Timestamp extraction is then limited to text starting with character 12 and ending with character 41.
Example
Examine this example event:
1989/12/31 16:00:00 Wed May 23 15:40:21 2007 ERROR UserManager - Exception thrown Ignoring unsupported search for eventtype: /doc sourcetype="access_combined" NOT eventtypetag=bot
To identify the timestamp as the second string of time information, May 23 15:40:21 2007
, configure the props.conf file like this:
[source::/Applications/splunk/var/spool/splunk] TIME_PREFIX = \d{4}\/\d{2}\/\d{2} \d{2}:\d{2}:\d{2} \w+\s MAX_TIMESTAMP_LOOKAHEAD = 21
This configuration instructs the Splunk platform to locate events that match the first timestamp construction, but to ignore that timestamp in favor of another timestamp that occurs within the following 21 characters, a number it gets from the MAX_TIMESTAMP_LOOKAHEAD
setting. The Splunk platform finds the second timestamp because it always occurs within that 21-character limit.
You can optimize the speed of timestamp extraction by setting the value of MAX_TIMESTAMP_LOOKAHEAD
to look only as far into an event as you need for the timestamp you want to extract. In this example, MAX_TIMESTAMP_LOOKAHEAD
is optimized to look just 21 characters into the event past the regular expression value.
Configure timestamp recognition | Configure advanced timestamp recognition with datetime.xml |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.3.0, 9.3.1, 8.1.0, 8.1.10, 8.1.11, 8.1.12
Feedback submitted, thanks!