Define an automatic lookup in Splunk Web

Manual lookups are applied to the results of a search when they are invoked with the lookup command. Automatic lookups are applied to all searches at search time.

Splunk software does not support nested automatic lookups.

Add a new lookup to run automatically

Prerequisites
Review the following topics:

• Lookups and the search-time operations sequence for field lookup restrictions
• Define a CSV lookup in Splunk Web
• Define an external lookup in Splunk Web
• Define a KV Store lookup in Splunk Web
• Define a geospatial lookup in Splunk Web
• An example lookup in Splunk Web

A lookup definition that you have defined previously.
Steps

1. In Splunk Web, select Settings > Lookups.
2. Under Actions for Automatic Lookups, click Add new.
3. Select the Destination app.
4. Give your automatic lookup a unique Name.
5. Select the Lookup table that you want to use in your fields lookup.
   This is the name of the lookup definition that you defined on the Lookup Definition page.
6. In the Apply to menu, select a host, source, or source type value to apply the lookup and give it a name in the named field.
7. Under Lookup input fields provide one or more pairs of input fields.
   The first field is the field in the lookup table that you want to match. The second field is a field from your events that matches the lookup table field. For example, you can have an ip_address field in your events that matches an ip field in the lookup table. So you would enter ip = ip_address in the automatic lookup definition.
8. Under Lookup output fields provide one or more pairs of output fields.
   The first field is the corresponding field that you want to output to events. The second field is the name that the output field should have in your events. For example, the lookup table may have a field named country that you may want to output to your events as ip_city. So you would enter country=ip_city in the automatic lookup definition.
   
   To avoid creating automatic lookup reference cycles, do not leave the Lookup output fields blank.
9. Select Overwrite field values to overwrite existing field values in events when the lookup runs. If you do not select this checkbox, the Splunk software does not apply the lookup to events where the output fields already exist.
   Note: This is equivalent to configuring your fields lookup in props.conf.
10. Click Save.

The Automatic lookup view appears, and the lookup that you have defined is listed.

If you have selected Overwrite field values, the automatic lookup lists with the keyword OUTPUT in its name. If you do not select Overwrite field values, the automatic lookup lists with OUTPUTNEW in its name.

Avoid defining automatic lookups with the _time, earliest, or latest fields set as output fields

If you define your automatic lookups with the _time, earliest or latest fields set as output fields, your searches might return unexpected results. For example, say your automatic lookup specifies _time as an output field, and then you run the following search in Splunk Web:

index=_internal source=*splunkd.log host=sh* earliest=-7d@d latest=now | stats count by host

Using the _time field as an output field for your automatic lookup might interfere with the ability of the Splunk platform to interpret the earliest and latest filters in the search. If that happens, the search will use the time period specified by the time range picker instead of the period you want. To avoid similar unexpected results, don't define automatic lookups with _time, earliest or latest as an output field.

Avoid creating automatic lookup reference cycles

You will receive warning messages for automatic lookup definitions that contain lookup reference cycles. A reference cycle occurs when lookup input and output fields end up being reused, either within the same lookup configuration, or among related lookup configurations.

For example, the following lookup configuration sets up a simple reference cycle where the type field appears as an input field and an output field. It is a case where the field that you are matching in your events is the same field that you are adding to your events.

• LOOKUP-meeting-type meeting_type_lookup object.type as type OUTPUTNEW meeting_type as type

You can accidentally set up more complex reference cycles between two or more related lookup configurations. For example, you might have a situation where multiple lookups combine to have fieldA ← fieldB ← fieldC ← fieldA.

Lookup reference cycles are often accidentally created when the Lookup output fields are left blank during the definition of an automatic lookup. When you leave Lookup output fields blank, the Splunk software uses all of the fields in the lookup table that are not the match fields as implicit output fields. Implicit output fields can easily create situations where the same field names appear in the match and output field sets. It can also set up reference cycles that involve multiple lookup configurations.

For example, say you have a lookup table named columns that contains five fields: column1, column2, column3 column4, and column5. Then you set up the following two lookup configurations that both leverage the columns lookup table:

• LOOKUP-col-testA columns column1 as column2 OUTPUT
• LOOKUP-col-testB columns column1 as column3 OUTPUTNEW column4, column5 as field5

When you consider that the implicit output fields for LOOKUP-col-testA are actually all of the fields from the column lookup table except column1 (meaning column2, column3 column4, and column5), you can see how this can cause these configurations to get tangled up with each other.

This table shows you the lookup reference cycles that different searches will encounter as a result of the way these automatic lookups have been configured:

Search	Reference cycle encountered
column2=*	column2 ← column2
column3=*	column3 ← column2 ← column2
field5=*	field5 ← column3 ← column2 ← column2

Each of these searches returns a lookup reference cycle warning message through the UI. The warning message tells you to inspect search.log for details and update lookup configurations to remove the reference cycle. When you go to search.log, you will find an extended warning message that identifies the reference cycle involved (such as field5 ← column3 ← column2 ← column2) and lists the configurations that require update.