Define an automatic lookup in Splunk Web
Manual lookups are applied to the results of a search when they are invoked with the lookup
command. Automatic lookups are applied to all searches at search time.
Splunk software does not support nested automatic lookups.
Add a new lookup to run automatically
Prerequisites
Review the following topics:
- Lookups and the search-time operations sequence for field lookup restrictions
- Define a CSV lookup in Splunk Web
- Define an external lookup in Splunk Web
- Define a KV Store lookup in Splunk Web
- Define a geospatial lookup in Splunk Web
- An example lookup in Splunk Web
A lookup definition that you have defined previously.
Steps
- In Splunk Web, select Settings > Lookups.
- Under Actions for Automatic Lookups, click Add new.
- Select the Destination app.
- Give your automatic lookup a unique Name.
- Select the Lookup table that you want to use in your fields lookup.
This is the name of the lookup definition that you defined on the Lookup Definition page. - In the Apply to menu, select a host, source, or source type value to apply the lookup and give it a name in the named field.
- Under Lookup input fields provide one or more pairs of input fields.
The first field is the field in the lookup table that you want to match. The second field is a field from your events that matches the lookup table field. For example, you can have anip_address
field in your events that matches anip
field in the lookup table. So you would enterip = ip_address
in the automatic lookup definition. - Under Lookup output fields provide one or more pairs of output fields.
The first field is the corresponding field that you want to output to events. The second field is the name that the output field should have in your events. For example, the lookup table may have a field namedcountry
that you may want to output to your events asip_city
. So you would entercountry=ip_city
in the automatic lookup definition.
To avoid creating automatic lookup reference cycles, do not leave the Lookup output fields blank. - Select Overwrite field values to overwrite existing field values in events when the lookup runs. If you do not select this checkbox, the Splunk software does not apply the lookup to events where the output fields already exist.
Note: This is equivalent to configuring your fields lookup inprops.conf
. - Click Save.
The Automatic lookup view appears, and the lookup that you have defined is listed.
If you have selected Overwrite field values, the automatic lookup lists with the keyword OUTPUT in its name. If you do not select Overwrite field values, the automatic lookup lists with OUTPUTNEW in its name.
Avoid defining automatic lookups with the _time, earliest, or latest fields set as output fields
If you define your automatic lookups with the _time
, earliest
or latest
fields set as output fields, your searches might return unexpected results. For example, say your automatic lookup specifies _time
as an output field, and then you run the following search in Splunk Web:
index=_internal source=*splunkd.log host=sh* earliest=-7d@d latest=now
| stats count by host
Using the _time
field as an output field for your automatic lookup might interfere with the ability of the Splunk platform to interpret the earliest and latest filters in the search. If that happens, the search will use the time period specified by the time range picker instead of the period you want. To avoid similar unexpected results, don't define automatic lookups with _time
, earliest
or latest
as an output field.
Avoid creating automatic lookup reference cycles
You will receive warning messages for automatic lookup definitions that contain lookup reference cycles. A reference cycle occurs when lookup input and output fields end up being reused, either within the same lookup configuration, or among related lookup configurations.
For example, the following lookup configuration sets up a simple reference cycle where the type
field appears as an input field and an output field. It is a case where the field that you are matching in your events is the same field that you are adding to your events.
LOOKUP-meeting-type meeting_type_lookup object.type as type OUTPUTNEW meeting_type as type
You can accidentally set up more complex reference cycles between two or more related lookup configurations. For example, you might have a situation where multiple lookups combine to have fieldA ← fieldB ← fieldC ← fieldA.
Lookup reference cycles are often accidentally created when the Lookup output fields are left blank during the definition of an automatic lookup. When you leave Lookup output fields blank, the Splunk software uses all of the fields in the lookup table that are not the match fields as implicit output fields. Implicit output fields can easily create situations where the same field names appear in the match and output field sets. It can also set up reference cycles that involve multiple lookup configurations.
For example, say you have a lookup table named columns
that contains five fields: column1
, column2
, column3
column4
, and column5
. Then you set up the following two lookup configurations that both leverage the columns
lookup table:
LOOKUP-col-testA columns column1 as column2 OUTPUT
LOOKUP-col-testB columns column1 as column3 OUTPUTNEW column4, column5 as field5
When you consider that the implicit output fields for LOOKUP-col-testA
are actually all of the fields from the column
lookup table except column1
(meaning column2
, column3
column4
, and column5
), you can see how this can cause these configurations to get tangled up with each other.
This table shows you the lookup reference cycles that different searches will encounter as a result of the way these automatic lookups have been configured:
Search | Reference cycle encountered |
---|---|
column2=*
|
column2 ← column2
|
column3=*
|
column3 ← column2 ← column2
|
field5=*
|
field5 ← column3 ← column2 ← column2
|
Each of these searches returns a lookup reference cycle warning message through the UI. The warning message tells you to inspect search.log for details and update lookup configurations to remove the reference cycle. When you go to search.log, you will find an extended warning message that identifies the reference cycle involved (such as field5 ← column3 ← column2 ← column2
) and lists the configurations that require update.
Define a time-based lookup in Splunk Web | Lookup example in Splunk Web |
This documentation applies to the following versions of Splunk Cloud Platform™: 9.2.2406, 9.0.2205, 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403 (latest FedRAMP release)
Feedback submitted, thanks!