Getting Data In

Configure advanced timestamp recognition with datetime.xml

Datetime.xml is a file that Splunk Enterprise uses to extract dates and timestamps from events as it indexes them. The file contains regular expressions that describe how Splunk Enterprise performs those extractions from the raw event data.

In nearly all cases, you do not need to make modifications to datetime.xml. When you configure timestamp recognition with props.conf, Splunk Enterprise uses datetime.xml to configure its timestamp processor and extract timestamps out of the events for your source, source type, or host. If the software is not able to process the timestamps in your event data, you can train Splunk Enterprise to extract the timestamps by making a custom version of datetime.xml.

Structure of datetime.xml

The datetime.xml file has the following structure:

• Code blocks that define individual elements of a time stamp, such as year, month, day, hour, and minute. The following example code block defines the regular expression that Splunk Enterprise uses to extract a literal month element (for example, Jan, Mar) out of event data:

<define name="_litmonth"  extract="litmonth">
<text><![CDATA[(?<![\d\w])(jan|\x{3127}\x{6708}|feb|\x{4E8C}\x{6708}|mar|\x{4E09}\x{6708}|apr|\x{56DB}\x{6708}|may|\x{4E94}\x{6708}|jun|\x{516D}\x{6708}|jul|\x{4E03}\x{6708}|aug|\x{516B}\x{6708}|sep|\x{4E5D}\x{6708}|oct|\x{5341}\x{6708}|nov|\x{5341}\x{3127}\x{6708}|dec|\x{5341}\x{4E8C}\x{6708})[a-z,\.;]*]]></text>
</define>

• Code blocks that use other elements that have been defined within the file. The following example code block defines the _time element which extracts hours, minutes, seconds, subseconds, period of day, and time zone:

<define name="_time" extract="hour, minute, second, subsecond, ampm, zone">
<text><![CDATA[(?<!\d)]]></text>
        <use name="_hour"/>
	<text><![CDATA[:]]></text>
        <use name="_minute"/>
	<text><![CDATA[:]]></text>
        <use name="_second"/> 
	<text><![CDATA[(?:(?: \d{4})?[:,\.](\d+))? {0,2}]]></text>
        <use name="_ampm"/>
	<text><![CDATA[ {0,2}]]></text>
        <use name="_zone"/>
	<text><![CDATA[(?!:\d)]]></text>
</define>

• Extraction pattern code blocks that define the order in which to attempt extracting times and dates from incoming event data. The timePatterns block defines the order in which Splunk Enterprise attempts to extract a timestamp, and the datePatterns block defines how to extract dates.

Each definition code block has one or more <text> definitions that contain a regular expression that Splunk Enterprise uses to extract the timestamp element.

Create or modify a custom datetime.xml

In nearly all cases, you do not need to modify datetime.xml. Instead, configure props.conf for timestamp extractions. See Edit timestamp properties in props.conf for instructions.

If Splunk Enterprise does not extract dates and times properly with props.conf, you might need to modify or substitute datetime.xml with a custom version. You can use the splunk train CLI command to take a sample of the timestamp data and generate code that you can use to create a custom datetime.xml that extracts your timestamp.

The splunk train CLI command is deprecated, but is still available to help you create patterns for datetime.xml based on your sample timestamp data.

After you create a pattern file with splunk train, you can make a copy of the default datetime.xml file and add your modifications to it, or you can create a new datetime.xml that only contains your custom timestamp definitions.

Never make modifications directly to $SPLUNK_HOME/etc/datetime.xml. Splunk Enterprise overwrites this file any time you upgrade, and any errors in the file that occur as the result of your changes can cause serious, lasting problems with data ingestion for both your custom source type and all other source types on the instance. If you want to make changes to the default file, save a copy to $SPLUNK_HOME/etc/system/local and make the changes there.

1. Create a sample timestamp pattern file
2. Run the splunk train CLI command against the file
3. Use the output to create a custom datetime.xml file
4. Reference the custom datetime.xml file in your timestamp configuration

Create a sample timestamp pattern file

1. From a prompt or PowerShell window, create a text file.
2. Paste in the sample of your timestamp data into this file.
3. Save the file and close it.
4. Change to the $SPLUNK_HOME/bin directory.

Run the splunk train CLI command against the file

1. Change to the $SPLUNK_HOME/bin directory:
   

   cd $SPLUNK_HOME/bin
   

2. Run the splunk train CLI command:
   

   ./splunk train dates
   

3. After the software asks the action you want to perform, type in L, l, or learn to perform the "learn" action.
4. Enter the path to the file that contains the timestamp sample.
5. Splunk Enterprise displays the first line of your sample and prompts you to enter values that represent the timestamp:
   

   ------------------------------------
   Interactively learning date formats.
   ------------------------------------
   INSTRUCTIONS: If a sample line does not have a timestamp, hit Enter.
   If it does have a timestamp, enter the timestamp separated by commas
   in this order: month, day, year, hour, minute, second, ampm, timezone.
   Use a comma as a placeholder for missing values.  For example, for a
   sample line like this "[Jan/1/08 11:56:45 GMT] login", the input 
   should be: "Jan, 1, 08, 11, 56, 45, , GMT" (note missing AM/PM).
   Spaces are optional.
   SAMPLE LINE 1:
           Tue Jul 10 21:23:06 PDT 2007 Received Trade 330 with detail user: user3456 date: date: 10Jul200721:
           23:06 action: sell 3583 MNAG @ 42
   --------------------------------------------------------------------------------
   Enter timestamp values as: month, day, year, hour, minute, second, ampm, timezone.
      >
   

   
   Enter values for month, day, year, hour, minute, second, period of day (am/pm), and time zone.
6. If the values are sufficient, Splunk software displays the following to show it has remembered the pattern:
   

   Learned pattern.
   ----------------------------------------------------------------------------------
   If you are satisfied that the timestamps formats have been learned, hit control-c.
   ----------------------------------------------------------------------------------
   

   
   Hit Ctrl-C if Splunk Enterprise has correctly learned the timestamp formats.
7. Splunk software displays text similar to the following:
   

   Patterns Learned. 
   It is highly recommended that you make changes to a copy of the default datetime.xml file.
   For example, copy "/Applications/splunk/etc/datetime.xml" to "/Applications/splunk/etc/system/local/datetime.xml", and work with that file.
   In that custom file, add the below timestamp definitions, and add the pattern names 
   to timePatterns and datePatterns list.
   For more details, see http://www.splunk.com/doc/latest/admin/TrainTimestampRecognition
   --------------------------------------------------------------------------------
   <define name="mycustom_date" extract="day,litmonth,year,">
   <text><![CDATA[:\d+\s\w+\s(\d+)\s(\w+)\s(\d+)]]></text>
   </define>
   <define name="mycustom_time" extract="hour,minute,second,ampm,">
   <text><![CDATA[(\d+):(\d+):(\d+)\s(\w+)]]></text>
   </define>
   ------------------------------------------------------
   What operation do you want to perform? (default=learn)
   ------------------------------------------------------
                              Enter choice: [Learn]/Test/Quit > 
   

   
   Review the pattern definitions in the output. If the definition for your timestamp sample looks the way that you want, quit the splunk train session by typing in Q, q, or quit. Otherwise, type in L, l, or learn again to attempt the training operation again.

Use the output to create a custom datetime.xml file

After you successfully train Splunk Enterprise to understand your custom timestamp, you must add the definition that splunk train generated to a custom version of datetime.xml.

You have two options to create this file:

• Add your timestamp definitions to an existing datetime.xml. This is the preferred method.
• Create a new datetime.xml that contains only your customized timestamp definitions. This option is better when the source type for your data is in a very strict format and Splunk Enterprise was incorrectly choosing a broader default format.

See "Examples of custom datetime.xml configuration" later in this topic.

1. Make a copy of datetime.xml in the $SPLUNK_HOME directory.
   

   cd $SPLUNK_HOME/etc
   cp datetime.xml etc/system/local/
   

   Never make edits to $SPLUNK_HOME/etc/datetime.xml. Always make a copy of this file and add your custom timestamp patters to the copy.

2. Open $SPLUNK_HOME/etc/system/local/datetime.xml for editing.
3. Copy the block of code that splunk train generated and that begins with define name, on its own line, into the file. This code block can go anywhere between the <datetime> and <timePatterns> entries. See "Examples of custom datetime.xml configuration" later in this topic for examples.
4. Within the <timePatterns> block, add a line that references the definition line you added earlier in the datetime.xml file.
5. Within the <datePatterns> code block, add the same line you added in the previous step.
6. Save the custom datetime.xml file and close it.

Reference the custom datetime.xml file in your timestamp configuration

After you build your custom datetime.xml file, you can reference it in props.conf to extract your custom timestamps. You can set a custom timestamp extraction pattern for any host, source, or source type.

1. In $SPLUNK_HOME/etc/system/local, create props.conf if it does not already exist.
2. Open props.conf in $SPLUNK_HOME/etc/system/local for editing.
3. Add a stanza for the host, source, or source type that requires the custom timestamp extraction, if it does not already exist.
4. Within this stanza, add a DATETIME_CONFIG setting that points to the custom datetime.xml, relative to the $SPLUNK_HOME directory. For example:
   

   [mysourcetype]
   DATETIME_CONFIG = /etc/system/local/datetime.xml
   MAX_TIMESTAMP_LOOKAHEAD = 128
   MAX_DAYS_AGO = 28
   

5. Repeat the previous steps as necessary for other hosts, sources, or source types that require the custom extraction.
6. Save props.conf and close it.
7. Restart Splunk software.
8. Confirm that timestamps are being extracted properly for the events that match the host, source, or source type that contains the custom timestamp extraction pattern.

Examples of custom datetime.xml configuration

The following blocks of code are examples of how to properly configure a custom datetime.xml file.

If splunk train generated code like the following:

<define name="mycustom_date" extract="day,litmonth,year,">
<text><![CDATA[:\d+\s\w+\s(\d+)\s(\w+)\s(\d+)]]></text>
</define>
<define name="mycustom_time" extract="hour,minute,second,ampm,">
<text><![CDATA[(\d+):(\d+):(\d+)\s(\w+)]]></text>
</define>

Example 1a: Modification of existing datatime.xml

Then you would add these definition blocks to an existing datetime.xml in $SPLUNK_HOME/etc/system/local that you copied previously:

<datetime>
 
<define name="mycustom_date" extract="day,litmonth,year,">
<text><![CDATA[:\d+\s\w+\s(\d+)\s(\w+)\s(\d+)]]></text>
</define>

<define name="mycustom_time" extract="hour,minute,second,ampm,">
<text><![CDATA[(\d+):(\d+):(\d+)\s(\w+)]]></text>
</define>

<... existing configurations removed for clarity ...>
 
<timePatterns>
      <use name="_time"/>
      <use name="_hmtime"/>
      <use name="_hmtime"/>
      <use name="_dottime"/>
      <use name="_combdatetime"/>
      <use name="_utcepoch"/>
      <use name="_combdatetime2"/>
      <use name="mycustom_time"/>
</timePatterns>

<datePatterns>
<use name="_usdate1"/> 
      <use name="_usdate2"/> 
      <use name="_isodate"/>
      <use name="_eurodate1"/> 
      <use name="_eurodate2"/> 
      <use name="_bareurlitdate"/> 
      <use name="_orddate"/>
      <use name="_combdatetime"/>
      <use name="_masheddate"/>
      <use name="_masheddate2"/>
      <use name="_combdatetime2"/>
      <use name="mycustom_date"/>
</datePatterns>

</datetime>

Example 1b: New datetime.xml with only your timestamp configuration

Or, you would create a new datetime.xml in $SPLUNK_HOME/etc/system/local, as follows:

<datetime>

<define name="mycustom_date" extract="day,litmonth,year,">
<text><![CDATA[:\d+\s\w+\s(\d+)\s(\w+)\s(\d+)]]></text>
</define>

<define name="mycustom_time" extract="hour,minute,second,ampm,">
<text><![CDATA[(\d+):(\d+):(\d+)\s(\w+)]]></text>
</define>

 <timePatterns>
       <use name="mycustom_date"/>
 </timePatterns>
 
 <datePatterns>
       <use name="mycustom_time"/> 
 </datePatterns>
 
 </datetime>

Example 2: Reference of new datetime.xml in props.conf for your custom source type

You would then reference the custom datetime.xml file in the configuration for your source type in props.conf, as follows:

$SPLUNK_HOME/etc/system/local/props.conf

[my_custom_sourcetype]
DATETIME_CONFIG=/etc/system/local/datetime.xml
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true