Splunk® Enterprise

Getting Data In

Download manual as PDF

Download topic as PDF

Configure timestamp assignment for events with multiple timestamps

If an event contains more than one timestamp, you can specify which timestamp is to be used for indexing. This is especially useful when indexing events that contain syslog host-chaining data.

Configure positional timestamp extraction by editing props.conf. For general information on editing props.conf for timestamps, see Configure timestamp recognition. If you have Splunk Enterprise and need to modify timestamp extraction, perform the configuration on your indexer machines or, if forwarding data, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run. If you have Splunk Cloud and need to modify timestamp extraction, use heavy forwarder and perform the configuration on the machines where the heavy forwarders run.


Configure positional timestamp extraction

To specify the position of the timestamp you want extracted, you add TIME_PREFIX and MAX_TIMESTAMP_LOOKAHEAD attributes to a props.conf stanza. By setting a regular expression for TIME_PREFIX, you specify the pattern of characters that indicates the point to start looking for the timestamp. Set a value for MAX_TIMESTAMP_LOOKAHEAD to specify how far into an event (past the TIME_PREFIX location) to look for the timestamp. By constraining lookahead, you can improve both accuracy and performance.

When TIME_PREFIX is set, Splunk software scans the event text for a match to its regular expression before it tries to extract a timestamp. The timestamping algorithm only looks for a timestamp in the text following the end of the first regular expression match. So if TIME_PREFIX is set to abc123, only the text following the first occurrence of abc123 is used for timestamp extraction.

TIME_PREFIX also sets the start point for MAX_TIMESTAMP_LOOKAHEAD; the lookahead starts after the matched portion of text in the TIME_PREFIX regular expression. For example, if TIME_PREFIX matches text through the first 11 characters of the event and the timestamp you want to extract is always within the next 30 characters, you can set MAX_TIMESTAMP_LOOKAHEAD=30. Timestamp extraction would be limited to text starting with character 12 and ending with character 41.

Example

Say you have an event that looks like this:

1989/12/31 16:00:00 Wed May 23 15:40:21 2007 ERROR UserManager - Exception thrown 
Ignoring unsupported search for eventtype: /doc sourcetype="access_combined" 
NOT eventtypetag=bot

To identify the timestamp as the second string of time information, May 23 15:40:21 2007, configure props.conf like this:

[source::/Applications/splunk/var/spool/splunk]
TIME_PREFIX = \d{4}\/\d{2}\/\d{2} \d{2}:\d{2}:\d{2} \w+\s 
MAX_TIMESTAMP_LOOKAHEAD = 21

This configuration instructs Splunk software to locate events that match the first timestamp construction, but ignore that timestamp in favor of a timestamp that occurs within the following 21 characters (a number it gets from the MAX_TIMESTAMP_LOOKAHEAD attribute). Splunk software will find the second timestamp because it always occurs within that 21-character limit.

Note: Optimize the speed of timestamp extraction by setting the value of MAX_TIMESTAMP_LOOKAHEAD to look only as far into an event as you need for the timestamp you want to extract. In this example, MAX_TIMESTAMP_LOOKAHEAD is optimized to look just 21 characters into the event past the regular expression value.

PREVIOUS
Configure timestamp recognition
  NEXT
Configure advanced timestamp recognition with datetime.xml

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 7.0.0, 7.0.1, 7.0.2


Comments

If I import a csv I see that TIMESTAMP_FIELDS is an attribute. Does this simply allow me to specify which field should be used if many fields contain time stamps? If there are spaces in the field name then should I enclose it in quotes?

Jbesant
July 16, 2015

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters