Knowledge Manager Manual

Use field lookups to add information to your events

The lookup feature lets you reference fields in an external CSV file that match fields in your event data. Using this match, you can enrich your event data by adding more searchable fields to them. You can base your field lookups on any field, including a temporal field, or on the output of a Python script.

This topic discusses how to use the Lookups page, located in Splunk Web, at Settings > Lookups, to:

• List existing lookup tables or upload a new file.
• Edit existing lookup definitions or define a new file-based or external lookup.
• Edit existing automatic lookups or configure a new lookup to run automatically.

For more details about lookups, see "Configure CSV and external lookups" and "Configure KV store lookups" in this manual.

List existing lookup tables or upload a new file

View existing lookup table files in Settings > Lookups > Lookup table files, or click "Add new" to upload more CSV files to use in your definitions for file-based lookups.

Note: CSV files with Pre-OSX OS9 and earlier "Classic Macintosh"-style line endings (aka just a carriage return, "\r") are not supported.

To upload new files:

1. Select a Destination app from the list.

Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.

2. Give your lookup table file a Name.

This will be the name you use to refer to the file in a lookup definition.

3. Browse for the CSV file to upload.

4. Click Save.

Edit existing lookup definitions or define a new file-based or external lookup

Use the Settings > Lookups > Lookup definitions page to define the lookup table or edit existing lookup definitions. You can specify the type of lookup (file-based or external) and whether or not it is time-based. Once you've defined the lookup table, you can invoke the lookup in a search (using the lookup command) or you can configure the lookup to occur automatically.

Note: This is equivalent to defining your lookup in transforms.conf.

Configure a time-based lookup

File-based and external lookups can also be time-based (or temporal), if the field matching depends on time information (a field in the lookup table that represents the timestamp).

To Configure a time-based lookup, select Configure time-based lookup, then specify the Name of the time field. You can also specify a strptime format for this time information and offsets for the time matching.

Include advanced options

Under Advanced options, you can specify:

• The minimum number of matches for each input lookup value.
• The maximum number of matches for each input lookup value.
• A default value to output if fewer than the minimum number of matches are present for a given input.

Edit existing automatic lookups or configure a new lookup to run automatically

Instead of invoking the lookup command when you want to apply a fields lookup to your events, you can set the lookup to run automatically. Use the Settings> Lookups > Automatic lookups page to edit or configure automatic lookups:

To edit an existing automatic lookup, select the lookup and modify the values in the fields displayed for that lookup.

To add a new new lookup to run automatically:

1. Select New in the Automatic lookups page:

2. Select the Destination app.

3. Select the Lookup table that you want use in your fields lookup.

This is the name of the lookup definition that you define on the Lookup Definition page.

4. In the Apply to menu, select a host, source, or source type value to apply the lookup and give it a name.

5. Under Lookup input fields provide one or more pairs of input fields.

The first field is the field in the lookup table that you want to match. The second field is a field from your events that should match the lookup table field. For example, you could have an ip_address field in your events that matches an ip field in the lookup table. So you would enter ip = ip_address in the automatic lookup definition.

6. Under Lookup output fields provide one or more pairs of output fields.

The first field is the corresponding field that you want to output to events. The second field is the name that the output field should have in your events. For example the lookup table may have a field named country that you may want to output to your events as ip_city.

7. You can also select Overwrite field values to overwrite the field values each time the lookup runs.

Note: This is equivalent to configuring your fields lookup in props.conf.

Example of HTTP status lookup

This examples walks through defining a static lookup that adds two informational fields, status_description and status_type, into your Web access events. This lets you search for the events you want when you might not know the specific error code. For example, instead of searching for all the server error codes, you can use status="Server Error".

Upload the lookup table to Splunk Enterprise

1. Download the http_status.csv file:

http_status.csv

Here's a sampling of the file:

status,status_description,status_type
100,Continue,Informational
101,Switching Protocols,Informational
200,OK,Successful
201,Created,Successful
202,Accepted,Successful
203,Non-Authoritative Information,Successful
...

2. Go back to the Search app, then select Settings > Lookups.

3. In the Lookups page, select Add new for Lookup table files.

4. In the Add new page,

• Select search for the destination app.
• Browse for the CSV file that you downloaded earlier.
• Name the lookup table http_status.
• Click Save.

After Splunk Enterprise saves the file, it takes you to the following view:

Now, let's go back to the Settings > Lookups view. To do this, click on the Lookups link in the page's breadcrumb. You can always use this to navigate back to a previous view.

Define the lookup

1. From Settings > Lookups, select Add new for Lookup definitions.

In the Add new page:

2. Select search for the Destination app.

3. Name your lookup definition http_status.

4. Select File-based under Type.

5. Click Save.

After Splunk Web saves your lookup definition, it takes you to the following page:

Notice there are some actions you can take on your lookup definition. Permissions lets you change the accessibility of the lookup table. You can Disable, Clone, and Move the lookup definition to a different app. Or, you can Delete the definition.

Once you define the lookup, you can use the lookup command to invoke it in a search or you can configure the lookup to run automatically.

Set the lookup to run automatically

1. Return to the Settings > Lookups view and select Add new for Automatic lookups.

In the Add new page:

2. Select search for the Destination app.

3. Name the lookup http_status.

4. Select http_status from the Lookup table drop down.

5. Apply the lookup to the sourcetype named access_combined.

6. Lookup input fields are the fields in our events that you want to match with the lookup table. Here, both are named status (the CSV column name goes on the left and the field that you want to match goes on the right):

7. Lookup output fields are the fields from the lookup table that you want to add to your events: status_description and status_type. The CSV column name goes on the left and the field that you want to match goes on the right.

8. Click Save.