
Configure timestamp recognition
Most events do not require special timestamp handling. Splunk software recognizes and extracts their timestamps correctly. However, with some sources and distributed deployments, you might need to configure how timestamps are extracted, to ensure they are formatted properly.
There are two ways to configure timestamp extraction:
- Use the "Set Sourcetype" page in Splunk Web to interactively adjust timestamps on sample data. Once you are happy with the results, you can save the changes to a new source type and then apply that source type to your data inputs. See The Set Sourcetype page.
- Edit props.conf directly, as explained in this topic.
If you have Splunk Enterprise and need to modify timestamp extraction, perform the configuration on your indexer machines or, if you forward data, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run. If you have Splunk Cloud and need to modify timestamp extraction, use heavy forwarders and perform the configuration on the machines where the heavy forwarders run.
The timestamp processor
The timestamp processor resides at $SPLUNK_HOME/etc/datetime.xml
by default. You do not need to touch this file normally, unless you work with unusual, custom timestamps. If you need to configure timestamp recognition in some way, you can make the necessary changes by setting props.conf
timestamp attributes, as described later in this topic.
If you have a custom timestamp that cannot be handled by configuring props.conf
, substitute your own timestamp processor with the DATETIME_CONFIG
attribute. This attribute specifies the file to be used by Splunk software for timestamp processing.
Edit timestamp properties in props.conf
To configure how Splunk software recognizes timestamps, edit props.conf. There are a number of attributes that pertain to timestamps. In particular, you can determine how Splunk software recognizes a timestamp by using the TIME_FORMAT
attribute to specify a strptime()
format for the timestamp.
You can also set other attributes that pertain to timestamps. This includes specifying where to look in an event for a timestamp, what time zone to use, or how to deal with timestamps of varying currency.
Edit the props.conf
file in $SPLUNK_HOME/etc/system/local/
or in your own custom application directory in $SPLUNK_HOME/etc/apps/
. For information on configuration files in general, see About configuration files in the Admin manual.
To set timestamp recognition, configure one or more of the timestamp attributes in props.conf
. Refer to the props.conf specification file for detailed information regarding these and other attributes.
Syntax overview
An overview of the syntax for the timestamp attributes follows:
[<spec>] DATETIME_CONFIG = <filename relative to $SPLUNK_HOME> TIME_PREFIX = <regular expression> MAX_TIMESTAMP_LOOKAHEAD = <integer> TIME_FORMAT = <strptime-style format> TZ = <POSIX time zone string> MAX_DAYS_AGO = <integer> MAX_DAYS_HENCE = <integer> MAX_DIFF_SECS_AGO = <integer> MAX_DIFF_SECS_HENCE = <integer>
In this syntax, <spec>
can be:
<sourcetype>
, the source type of an event.host::<host>
, where<host>
is the host value for an event.source::<source>
, where<source>
is the source value for an event.
If an event contains data that matches the value of <spec>
, then the timestamp rules specified in the stanza apply to that event. You can have multiple stanzas, to handle different <spec>
values.
Timestamp validity attributes and their impact on events
By default, all events are indexed, unless you specifically filter out events through other means.
When you set the MAX_DAYS_AGO
, MAX_DAYS_HENCE
, MAX_DIFF_SECS_AGO
, or MAX_DIFF_SECS_HENCE
attributes in a stanza, and event timestamps fall outside of the parameters of the attributes, then the software uses the following algorithm to determine the timestamp:
- The software uses the timestamp of the previous event to assign the timestamp of the current event.
- If the timestamp of the previous event can't be determined, then the software uses the current index time to assign a timestamp to the event.
Events are not dropped if they fall outside of the parameters of these attributes.
Timestamp attributes
You can set the following timestamp attributes with props.conf
.
Attribute | Description | Default |
---|---|---|
DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>
|
Specify a file to use to configure the Splunk timestamp processor.
Under normal circumstances, you do not need to create your own timestamp processor file or modify the default Set Set Note: Both |
$SPLUNK_HOME/etc/datetime.xml
|
TIME_PREFIX = <regular expression>
|
When set, Splunk software uses the specified regular expression to looks for a match before attempting to extract a timestamp. The timestamp algorithm only looks for a timestamp in the event text that follows the end of the first regular expression match.
You should use a regular expression that points exactly before your event's timestamp. For example, if the timestamp follows the phrase If the |
empty string |
MAX_TIMESTAMP_LOOKAHEAD = <integer>
|
Specify how far (how many characters) into an event Splunk software should look for a timestamp.
This constraint is applied starting from the location positioned by TIME_PREFIX. For example, if TIME_PREFIX positions a location 11 characters into the event, and MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction will be constrained to characters 11 through 20. If set to 0 or -1, the length constraint for timestamp recognition is effectively disabled. This can have negative performance implications which scale with the length of input lines (or with event size when LINE_BREAKER is redefined for event splitting). |
128 characters |
TIME_FORMAT = <strptime-style format>
|
Specifies a strptime() format string to extract the timestamp.
For best results, the If |
empty string |
TZ = <timezone_identifier>
|
The time zone of an event is determined as follows:
For more details and examples, see Specify time zones of timestamps in this manual. |
empty string |
TZ_ALIAS = <key=value>[,<key=value>]...
|
Provides admin-level control over how time zone strings extracted from events are interpreted. For example, EST can mean Eastern (US) Standard Time or Eastern (Australian) Standard Time. There are many other three letter time zone acronyms with multiple expansions.
There is no requirement to use Has no effect on the The setting is a list of The key is matched against the text of the time zone specifier of the event, and the value is the time zone specifier to use when mapping the timestamp to UTC/GMT. The value is another Example: |
not set |
MAX_DAYS_AGO = <integer>
|
Specifies the maximum number of days in the past, from the current date, that an extracted date can be valid.
For example, if The maximum settable number of days in the past is 10951. |
2000 days
Note: If you have data that is more than 2000 days old, increase this setting. |
MAX_DAYS_HENCE = <integer>
|
Specifies the maximum number of days in the future from the current date that an extracted date can be valid.
For example, if Note: False positives are less likely with a tighter window. Change this attribute with caution. If your servers have the wrong date set or are in a time zone that is one day ahead, set this value to at least 3. This allows timestamp extractions that are up to a day in the future. The maximum settable number of days is 10950. |
2 days |
MAX_DIFF_SECS_AGO = <integer>
|
If the event timestamp is more than <integer> seconds before the previous timestamp, Splunk software accepts it only if it has the same time format as the majority of timestamps from the source.
If your timestamps are wildly out of order, consider increasing this value. The maximum settable number of seconds is 2147483646. |
3600 seconds (1 hour) |
MAX_DIFF_SECS_HENCE = <integer>
|
If the event's timestamp is more than <integer> seconds after the previous timestamp, Splunk only accepts it if it has the same time format as the majority of timestamps from the source.
If your timestamps are wildly out of order, or if you have logs that are written less than once a week, consider increasing this value. The maximum settable number of seconds is 2147483646. |
604800 seconds (1 week) |
Enhanced strptime() support
Use the TIME_FORMAT
attribute in props.conf
to configure timestamp parsing. This attribute takes a strptime()
format string, which it uses to extract the timestamp.
Splunk software implements an enhanced version of Unix strptime() that supports additional formats, allowing for microsecond, millisecond, any time width format, and some additional time formats for compatibility. The additional formats include:
%N | For GNU date-time nanoseconds. Specify any sub-second parsing by providing the width: %3N = milliseconds, %6N = microseconds, %9N = nanoseconds. |
%Q,%q | For milliseconds, microseconds for Apache Tomcat. %Q and %q can format any time resolution if the width is specified. |
%I | For hours on a 12-hour clock format. If %I appears after %S or %s (like "%H:%M:%S.%l"), it takes on the log4cpp meaning of milliseconds. |
%+ | For standard Unix date format timestamps. |
%v | For BSD and OSX standard date format. |
%Z | The time zone abbreviation (nothing if there is no time zone information.) |
%z, %:z, %::z | The time zone offset designator in International Organization for Standardization (ISO) 8601 format (for example, -0800 for PST, +0000 for GMT, or nothing if the time zone cannot be determined.) Use %:z if the timestamp offset contains hours and minutes (for example, -08:00 ) and %::z if the timestamp offset contains hours, minutes, and seconds (for example, -08:00:00 .)
|
%o | For AIX timestamp support (%o used as an alias for %Y). |
%p | The locale's equivalent of AM or PM. (Note: there may be none.) |
%s | Epoch (10 digits) |
Note: A strptime() expression that ends with a literal dot and subsecond specifier such as %Q, %q, %N treats the terminal dot and conversion specifier as optional. If the .subseconds portion is absent from the text, the timestamp is still extracted.
strptime() format expression examples
Here are some sample date formats, with the strptime()
expressions that handle them:
1998-12-31 | %Y-%m-%d |
98-12-31 | %y-%m-%d |
1998 years, 312 days | %Y years, %j days |
Jan 24, 2003 | %b %d, %Y |
January 24, 2003 | %B %d, %Y |
1397477611.862 | %s.%3N |
Note: Splunk software does not recognize non-English month names in timestamps. If you have an app that writes non-English month names to log files, reconfigure the app to use numerical months, if possible.
Examples
Your data might contain an easily recognizable timestamp, such as:
...FOR: 04/24/07 PAGE 01
...
To extract that timestamp, add this stanza in props.conf
:
[host::foo] TIME_PREFIX = FOR: TIME_FORMAT = %m/%d/%y
Another example that includes time zone information:
…Valid_Until=Thu Dec 31 17:59:59 GMT-06:00 2020
To extract the timestamp, add this to props.conf
:
[host::bar] TIME_PREFIX = Valid_Until= TIME_FORMAT = %a %b %d %H:%M:%S %Z%z %Y
Your data might contain other information that is parsed as timestamps, for example:
...1989/12/31 16:00:00 Wed May 23 15:40:21 2007
...
Splunk software extracts the date as Dec 31, 1989, which is not useful. In this case, configure props.conf
to extract the correct timestamp from events from host::foo
:
[host::foo] TIME_PREFIX = \d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \w+\s TIME_FORMAT = %b %d %H:%M:%S %Y
This configuration assumes that all timestamps from host::foo
are in the same format. Configure your props.conf
stanza to be as granular as possible to avoid potential timestamping errors.
For more information on extracting the correct timestamp from events containing multiple timestamps, see Configure timestamp assignment for events with multiple timestamps.
Configure timestamps for specific needs
You can use the attributes described in this topic to configure the timestamp extraction processor for some specialized purposes, such as:
- To apply time zone offsets.
- To pull the correct timestamp from events with more than one timestamp.
- To improve indexing performance.
Configure how timestamps appear in search results
You can use your browser locale setting to configure how Splunk Web displays timestamps in search results. For information on setting the browser locale, see User language and locale.
Reconfigure how timestamps appear in raw data
Even though Splunk software uses the browser locale to configure how timestamps appear in search results, the raw data still remains in its original format. You might want to change this so that the data format is standardized in both raw data and search results. Do this with props.conf
and transforms.conf
. Here is an example:
Assume the timestamp data in the raw event looks like this:
06/07/2011 10:26:11 PM
but you want it to look like this (to correspond with how it appears in search results):
07/06/2011 10:26:11 PM
This example shows briefly how you can use props.conf
and transforms.conf
to transform the timestamp in the raw event.
In transforms.conf
, add this stanza:
[resortdate] REGEX = ^(\d{2})\/(\d{2})\/(\d{4})\s([^/]+) FORMAT = $2/$1/$3 $4 DEST_KEY = _raw
In props.conf
, add this stanza, where <spec>
qualifies your data:
[<spec>] TRANSFORMS-sortdate = resortdate
Answers
Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has around timestamp recognition and configuration.
PREVIOUS How timestamp assignment works |
NEXT Configure timestamp assignment for events with multiple timestamps |
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 8.0.0, 8.0.1
Comments
Where should i put timestamp configurations?
1. If you are using INDEXED_EXTRACTIONS = <.. > then put it to the server where INDEXED_EXTRACTION is configured.
2. If you are using Heavy forwarder then put it in the heavy forwarder
3. If you are using Indexer only then put it in the indexers.
"For GNU date-time nanoseconds." The only thing GNU here is the %N convention. "For milliseconds, microseconds for Apache Tomcat." This is not for apache tomcat. It's just accepting the %Q and %q conventions used by apache tomcat. "For standard Unix date formats". Unix does not have a standard date format. Unix "time_t" or seconds from epoch are what you're looking for here. Saying %v is "standard bsd osx" is unhelpful. Say " 1-Jan-2012", though I don't see the point of ever using this. %Z behavior underdefined . what does "nothing" mean? Probably it's UTC if not recognized. :-/
Hi Lguinn,
I'm researching this now, but I would like to think that yes, we do, since we support an enhanced version of the format.
I will confirm and update here. Thanks for your patience.
Does the Splunk implementation of strptime support %t and %n? Also, I think the specification for these has changed within the last few years. What rev of the strptime standard does Splunk support?
Hi BradTaylor,
You're right, it does look like the examples you show are missing the strptime token for day of week. I think the second does account for the day of week because of the \w+\s regular expression, but the first one is a no-brainer. I'll fix this.
Thanks,
-Malcolm
For these two examples:
Valid_Until=Thu Dec 31 17:59:59 GMT-06:00 2020
...
[host::bar]
TIME_PREFIX = Valid_Until=
TIME_FORMAT = %b %d %H:%M:%S %Z%z %Y
AND
...1989/12/31 16:00:00 Wed May 23 15:40:21 2007...
...
[host::foo]
TIME_PREFIX = \d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \w+\s
TIME_FORMAT = %b %d %H:%M:%S %Y
The timestamps both have Day of Week tokens. But there is no %a leading strptime token to capture that. (Maybe it's not required in the second example because of the \w+\s regex pattern... fair enough. But what about the first one?)
HI Graham Hannington,
I am still researching information on your previous comment. I should have answers for you shortly (though it will likely not be until after .conf.)
I'll make updates to the topic again at that time based on that information.
Some nits in the description of "%z, %:z, %::z":
- Missing closing parenthesis.
- Misplaced periods next to closing parentheses. Unless parentheses enclose a complete sentence, the period should follow the closing parenthesis.
- "ISO-8601-style"? All those hyphens! "Numeric"? I get that you're distinguishing these formats from the uppercase %Z, but since you namecheck ISO 8601, perhaps use its terminology. "ISO 8601 zone designator"? (And I have "Hyphenate compound adjectives" tattooed in blackletter across my midriff!)
- Consistency: timezone (one word) versus time zone (two: this is the term that ISO 8601 uses).
All minor, but this stuff sticks out to me like dogs' balls. Yes, my problem, but these comments give me the opportunity to vent ;-).
Suggestion: rewrite in a tabular, reference style, as per other formats. Rather than nesting the descriptions, three separate rows. Or reuse:
https://docs.splunk.com/Documentation/Splunk/6.4.3/SearchReference/Commontimeformatvariables
In practice, the %6N (note the width: 6 decimal places, for microsecond precision) in
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%6N%:z
also recognizes time stamps with millisecond precision (3 decimal places).
That is, Splunk correctly recognizes both of the following time stamps (and Splunk Web displays them correctly):
2016-09-23 08:00:00.123+08:00
2016-09-23 08:00:00.123456+08:00
with or without the trailing zone designator (+08:00)
Shrug. So what?
Here's why it matters to me: I'm documenting example .conf stanzas for ingesting data from an application that I'm helping to develop. Users can configure that application to output time stamps with millisecond or microsecond precision. I was wondering whether I'd have to warn users about specifying the correct "sub-second width", %3N or %6N, in TIME_FORMAT. Turns out I don't have to: %6N works for both.
Splunk, please feel free to correct me.
Hi all,
I've made updates to the page to better clarify the %Z and %[:][:]z strptime() settings. Zerone is correct in that the colons in the %z setting represent how to split the timezone offset. The %:::z setting is a typo and has been removed.
%z works for +1000 (-1000), will stop at +10 for time offset of +10:00 or +10:00:00
%:z works for +1000 (-1000) and +10:00 (-10:00), will stop at +10:00 for time offset of +10:00:00
%::z works for +1000 (-1000) and +10:00:00 (-10:00:00), will error for (+10:00 or -10:00), will stop at +10:00:00 for time offset of +10:00:00:00
Could you please provide more detail in this topic, or a link to details elsewhere, on the formats (the IEEE/Open Group doc refers to these as "conversion specifications") for time zones?
I discovered in Splunk Answers that the following format:
%:z
(note the single colon)
was what I needed to match an ISO 8601 extended format zone designator such as:
+08:00
but this Splunk docs topic does not mention that specific format (%:z, with the single colon).
The topic *does* mention "GNU libc support", but the topic at the following link:
http://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html
only mentions %z and %Z, not any variants with colons. (That is, %z matches a "basic format" ISO 8601 zone designator such as +0800, not an "extended format" designator with a colon separator.)
Hi Woodcock,
Splunk Enterprise never drops events if event timestamps are outside of the acceptable window. The default for MAX_DAYS_AGO is 2000 which means that timestamps can be askew for nearly 5 1/2 years before the software has to decide what to do with the events. In nearly all cases, it uses the timestamp of the previous event. If it can't determine the timestamp of the previous event, it uses the time that the event was indexed. The default for MAX_DAYS_HENCE is 2, but the resolution logic is the same.
For MAX_DAYS_AGO and MAX_DAYS_HENCE, when the timestamp is outside of the acceptable window, what happens when "the date is ignored"? Does it:
1: Use _indextime?
2: Use the timestamp of the previous event?
3: Drop the event entirely?
This detail should be included in the documentation.
how do you assign TIME to a field called "time" in your sourcetype definition?