Configure timestamp assignment for events with multiple timestamps

If an event contains more than one timestamp, you can specify which timestamp the event is to use for indexing. Configuring the timestamp is especially useful when you are indexing events that contain syslog host-chaining data.

Configure positional timestamp extraction by editing the props.conf configuration file. While there is limited ability to configure timestamp extraction in Splunk Web, you can achieve the best results by using configuration files on a heavy forwarder. See props.conf for more information on this configuration file.

Splunk Cloud Platform

On a Splunk Cloud Platform instance, you can configure timestamps on a heavy forwarder after you configured that forwarder to send data to the Splunk Cloud Platform instance. If you aren't using a heavy forwarder but have access to the Edge Processor solution, you can configure timestamps using Edge Processors if necessary. See Extract timestamps from event data using an Edge Processor in the Use Edge Processors manual. For general information on editing the props.conf file for timestamps, see Configure timestamp recognition.

Splunk Enterprise

To modify timestamp extraction, perform the configuration on your indexer machines or, if you are forwarding data, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run.

Prerequisites

• Only users with file system access, such as system administrators, can <describe the customization> using configuration files.
• Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.
• You can have configuration files with the same name in your default, local, and app directories. Read Where you can place (or find) your modified configuration files in the Splunk Enterprise Admin Manual.

Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make changes to the files in the local directory.

Steps

1. Open or create a local <props.conf_file> file at $SPLUNK_HOME/etc/system/local.
2. Add TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD settings to a stanza in the props.conf file.
   By setting a regular expression for TIME_PREFIX, you specify the pattern of characters that indicates the point to start looking for the timestamp.
3. Set a value for the MAX_TIMESTAMP_LOOKAHEAD setting to specify how far into an event past the TIME_PREFIX location to look for the timestamp.
   By constraining the amount of time to look ahead, you can improve both the accuracy and performance in determining and extracting the timestamp.

When you set TIME_PREFIX setting, the Splunk platform scans the event text for a match to its regular expression before it tries to extract a timestamp. The timestamp algorithm looks for a timestamp in the text following the end of the first regular expression match. For example, if the TIME_PREFIX setting is set to abc123, only the text following the first occurrence of abc123 is used for timestamp extraction.

The TIME_PREFIX setting also sets the start point for the MAX_TIMESTAMP_LOOKAHEAD setting. The lookahead starts after it finds the matched portion of text in the TIME_PREFIX regular expression. For example, if TIME_PREFIX matches text through the first 11 characters of the event and the timestamp you want to extract is always within the next 30 characters, you can set MAX_TIMESTAMP_LOOKAHEAD=30. Timestamp extraction is then limited to text starting with character 12 and ending with character 41.

Example

Examine this example event:

1989/12/31 16:00:00 Wed May 23 15:40:21 2007 ERROR UserManager - Exception thrown 
Ignoring unsupported search for eventtype: /doc sourcetype="access_combined" 
NOT eventtypetag=bot

To identify the timestamp as the second string of time information, May 23 15:40:21 2007, configure the props.conf file like this:

[source::/Applications/splunk/var/spool/splunk]
TIME_PREFIX = \d{4}\/\d{2}\/\d{2} \d{2}:\d{2}:\d{2} \w+\s 
MAX_TIMESTAMP_LOOKAHEAD = 21

This configuration instructs the Splunk platform to locate events that match the first timestamp construction, but to ignore that timestamp in favor of another timestamp that occurs within the following 21 characters, a number it gets from the MAX_TIMESTAMP_LOOKAHEAD setting. The Splunk platform finds the second timestamp because it always occurs within that 21-character limit.

You can optimize the speed of timestamp extraction by setting the value of MAX_TIMESTAMP_LOOKAHEAD to look only as far into an event as you need for the timestamp you want to extract. In this example, MAX_TIMESTAMP_LOOKAHEAD is optimized to look just 21 characters into the event past the regular expression value.