Getting Data In

How timestamp assignment works

Splunk cares a lot about timestamps. Splunk uses timestamps to correlate events by time, to create the timeline histogram in Splunk Web, and to set time ranges for searches.

Splunk assigns timestamps to events at index time. It usually assigns timestamp values automatically, using information in the raw event data. If an event doesn't contain an explicit timestamp, Splunk attempts to assign a timestamp value through other means. For some data, Splunk might need your help to tell it how to recognize the timestamps.

Splunk stores timestamp values in the _time field (in UTC time format).

Timestamp processing is one of the key steps in event processing. For more information on event processing, see the chapter in this manual called "Configure event processing".

How Splunk assigns timestamps

Splunk uses the following precedence rules to assign timestamps to events:

1. Splunk looks for a time or date in the event itself using an explicit TIME_FORMAT, if provided. You configure TIME_FORMAT in props.conf.

2. If no TIME_FORMAT was configured for the data, Splunk attempts to automatically identify a time or date in the event itself. It uses the event's source type (which includes TIME_FORMAT information) to try to find the timestamp.

3. If an event doesn't have a time or date, Splunk uses the timestamp from the most recent previous event of the same source.

4. If no events in a source have a date, Splunk tries to find one in the source name or file name. (This requires that the events have a time, even though they don't have a date.)

5. For file sources, if no date can be identified in the file name, Splunk uses the file's modification time.

6. As a last resort, Splunk sets the timestamp to the current system time when indexing each event.

Note: Splunk can only extract dates from a source, not times. If you need to extract a time from a source, use a transform.

Configure timestamps

Most events don't require any special timestamp handling. Splunk automatically recognizes and extracts their timestamps. However, for some sources and distributed deployments, you might need to configure how Splunk extracts timestamps, so that they format properly.

There are two ways to configure timestamp extraction:

• Use the data preview feature to interactively adjust timestamps on sample data. Once you're happy with the results, you can save the changes to a new source type and then apply that source type to your data inputs. See the chapter "Preview your data".

• Edit props.conf directly. For more information, read Configure timestamp recognition.

You can also configure Splunk's timestamp extraction processor to:

• Apply time zone offsets.
• Pull the correct timestamp from events with more than one timestamp.
• Improve indexing performance.

Considerations when adding data from new inputs

If you index some data from a new input and then discover that you need to adjust the timestamp extraction process, you will need to re-index that data once you've made the configuration changes. Therefore, it's a good idea to preview your data, as described in the chapter "Preview your data".

Alternatively, you can test new data inputs in a test instance of Splunk (or just in a separate index on the production Splunk instance) before adding data to your production Splunk instance. That way, if you need to make adjustments, you can easily clean out the data and re-index it until you get it right.