Splunk Cloud Platform

Knowledge Manager Manual

Configure a time-based lookup

If your lookup table has a field that represents time, you can use it to create a time-based lookup. This is also referred to as a temporal lookup. You can configure all four lookup types as time-based lookups.

Simple time-based lookups attempt to match the event timestamp with the timestamp of a record in the lookup table, and then perform operations like adding one or more fields to the event from the matched record.

You can also define time-bound lookups, which use the event time to define a range of time within which to match lookup records. For example, you could create a time-bound lookup that matches the first lookup table record with a timestamp that falls within 10 seconds before the event timestamp.

Defining time-based lookups

To create a simple time-based lookup, add the following lines to your lookup stanza in transforms.conf:

time_field = <field_name>
time_format = <string>

Here are the definitions of these settings.

Setting Description Default
time_field Identifies the field in the lookup table that represents the timestamp. The search processor applies the first matching entry in descending order.

When time_field is present in a saved search stanza, max_matches = 1 by default. For more information about max_matches see Add field matching rules to your lookup configuration.
Defaults to an empty string, because lookups are not time-based by default.
time_format Specifies the strptime() format of the time_field attribute. You can use some nonstandard date-time strptime() formats. See the material about enhanced strptime() support in Configure timestamp recognition in the Getting Data In Manual. %s.%Q

This is the Unix epoch time value in seconds (%s), with optional milliseconds (%Q).

Defining time-bound lookups

To create a time-bound lookup, add these optional settings to your time-based lookup configuration:

max_offset_secs = <integer>
min_offset_secs = <integer>

Here are the definitions of these settings:

Setting Description Default
max_offset_secs The maximum amount of time in seconds that an event timestamp can be later than the lookup record timestamp, for a match to occur. 2000000000 (effectively no default)
min_offset_secs The minimum amount of time in seconds that an event timestamp can be later than the lookup record timestamp, for a match to occur. 0

The max_offset_secs and min_offset_secs settings define the earliest and latest times within which the search processor can search for matching records in the lookup table. The search processor calculates the earliest and latest time values from the event time like this:

earliest = event timestamp - max_offset_secs
latest = event timestamp - min_offset_secs

Within this window of time, the search processor applies a match in descending order of time up to the point where we get max_matches number of matches for that event. If max_matches is not set, it defaults to 1. For more information about max_matches see Add field matching rules to your lookup configuration.

Time-based lookup example

Here's an example of a CSV lookup that uses DHCP logs to identify users on a network based on their IP address and the timestamp. The DHCP logs are in a file, dhcp.csv, which contains the timestamp, IP address, and the user's name and MAC address.

Prerequisities

Steps

  1. In a transforms.conf file, put:
    [dhcpLookup]
    filename = dhcp.csv
    time_field = timestamp
    time_format = %d/%m/%y %H:%M:%S
    
  2. In a props.conf file, make the lookup automatic:
    [dhcp]
    LOOKUP-table = dhcpLookup ip mac OUTPUT user
    
  3. Save your file changes.

If you wanted to turn this into a time-bound lookup, you could add the following settings to the [dhcpLookup] stanza in transforms.conf:

max_offset_secs = 10
min_offset_secs = 0

This would cause the lookup to match events to the first lookup table record with a timestamp that falls within a range of time bound by the event timestamp and ten seconds before the event timestamp.

Last modified on 11 November, 2021
Add field matching rules to your lookup configuration   Make your lookup automatic

This documentation applies to the following versions of Splunk Cloud Platform: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403 (latest FedRAMP release), 9.2.2406


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters