Splunk® Enterprise

Getting Data In

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Configure timestamp assignment for events with multiple timestamps

If an event contains more than one timestamp, you can specify which timestamp Splunk uses. This is especially useful when indexing events that contain syslog host-chaining data.

Configure positional timestamp extraction by editing props.conf. For general information on editing props.conf for timestamps, see "Configure timestamp recognition".

Configure positional timestamp extraction

Configure Splunk to recognize a timestamp anywhere in an event by adding TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD attributes to a props.conf stanza. By setting a regex value for TIME_PREFIX, you tell Splunk what pattern of characters indicate the point to start looking for the timestamp. Set a value for MAX_TIMESTAMP_LOOKAHEAD to tell Splunk how far into an event (past the TIME_PREFIX location) to look for the timestamp. By constraining lookahead, you can improve both accuracy and performance.

When TIME_PREFIX is set, Splunk scans the event text for a match to its regex before it tries to extract a timestamp. Splunk's timestamping algorithm only looks for a timestamp in the text following the end of the first regex match. So if TIME_PREFIX is set to abc123, only the text following the first occurrence of abc123 is used for timestamp extraction.

TIME_PREFIX also sets the start point for MAX_TIMESTAMP_LOOKAHEAD; the lookahead starts after the matched portion of text in the TIME_PREFIX regex. For example, if TIME_PREFIX matches text through the first 11 characters of the event and the timestamp you want to extract is always within the next 30 characters, you can set MAX_TIMESTAMP_LOOKAHEAD=30. Timestamp extraction would be limited to text starting with character 12 and ending with character 41.

Example

Say you have an event that looks like this:

1989/12/31 16:00:00 Wed May 23 15:40:21 2007 ERROR UserManager - Exception thrown 
Ignoring unsupported search for eventtype: /doc sourcetype="access_combined" 
NOT eventtypetag=bot

To identify the timestamp as the second string of time information, May 23 15:40:21 2007, configure props.conf like this:

[source::/Applications/splunk/var/spool/splunk]
TIME_PREFIX = \d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \w+\s 
MAX_TIMESTAMP_LOOKAHEAD = 21

This configuration instructs Splunk to locate events that match the first timestamp construction, but ignore that timestamp in favor of a timestamp that occurs within the following 21 characters (a number it gets from the MAX_TIMESTAMP_LOOKAHEAD attribute). Splunk will find the second timestamp because it always occurs within that 21-character limit.

Note: Optimize the speed of timestamp extraction by setting the value of MAX_TIMESTAMP_LOOKAHEAD to look only as far into an event as needed for the timestamp you want to extract. In this example, MAX_TIMESTAMP_LOOKAHEAD is optimized to look just 21 characters into the event past the regex value.

PREVIOUS
Configure timestamp recognition
  NEXT
Specify time zones of timestamps

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18


Comments

I am trying to use the TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD but it is not working. Is there a way to troubleshoot the regex extraction syntax for TIME_PREFIX?

Tablewski
July 26, 2011

Hi Deinspanjer! Actually the regex in TIME_PREFIX is correct; it's just that we probably didn't do the best job of explaining what TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD do. I've updated this page to better explain how these two attributes work together to help you locate a timestamp *within* an event by first matching (and then disregarding) the text at the start of the event.

Mness
March 30, 2011

The TIME_PREFIX listed here doesn't match the example.

Deinspanjer
March 30, 2011

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters