Splunk® Enterprise

Knowledge Manager Manual

Download manual as PDF

Download topic as PDF

Use lookups to add fields to your events

The lookup feature lets you reference fields in an external CSV file that match fields in your event data. Using this match, you can enrich your event data by adding more searchable fields to them. You can base your field lookups on any field, including a temporal field, or on the output of a Python script. CSV inline lookup table files and inline lookup definitions that use CSV files are both dataset types.

This topic discusses how to use the Lookups page, located in Splunk Web, at Settings > Lookups, to:

  • List existing lookup tables or upload a new file.
  • Edit existing lookup definitions or define a new file-based or external lookup.
  • Edit existing automatic lookups or configure a new lookup to run automatically.

For more details about lookups, see Configure CSV and external lookups and Configure KV store lookups in this manual.

Lookups and the search-time operations sequence

When you run a search, Splunk software runs several operations to derive knowledge objects and apply them to events returned by the search. Splunk software performs these operations in a specific sequence.

Search-time operations order

Field lookups come sixth in the order of search-time operations, before event types but after calculated fields.

Restrictions

Splunk software processes lookups belonging to a specific host, source, or sourcetype in ASCII sort order. Lookup configurations can reference fields that are added to events by field extractions, field aliases, and calculated fields, but cannot reference event types and tags.

For more information

See search time operations sequence for more information on the order of search-time operations.

List existing lookup tables or upload a new file

View existing lookup table files in Settings > Lookups > Lookup table files, or click "Add new" to upload more CSV files to use in your definitions for file-based lookups.

Note: CSV files with Pre-OSX OS9 and earlier "Classic Macintosh"-style line endings (aka just a carriage return, "\r") are not supported.

Prerequisites

To upload new files:

  1. Select a Destination app from the list. Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
  2. Give your lookup table file a Name. This will be the name you use to refer to the file in a lookup definition.
  3. Browse for the CSV file to upload.
  4. Click Save.

Edit existing lookup definitions or define a new file-based or external lookup

Use the Settings > Lookups > Lookup definitions page to define the lookup table or edit existing lookup definitions. You can specify the type of lookup (file-based or external) and whether or not it is time-based. Click the Advanced options checkbox to see additional options for your definition. Once you've defined the lookup table, you can invoke the lookup in a search (using the lookup command) or you can configure the lookup to occur automatically.

This is equivalent to defining your lookup in transforms.conf.

Configure a time-based lookup

File-based and external lookups can also be time-based (or temporal), if the field matching depends on time information (a field in the lookup table that represents the timestamp).

To Configure a time-based lookup, select Configure time-based lookup, then specify the Name of the time field. You can also specify a strptime format for this time information and offsets for the time matching.

Include advanced options

Under Advanced options, you can specify:

  • The minimum number of matches for each input lookup value.
  • The maximum number of matches for each input lookup value.
  • A default value to output if fewer than the minimum number of matches are present for a given input.

Edit existing automatic lookups or configure a new lookup to run automatically

Instead of invoking the lookup command when you want to apply a fields lookup to your events, you can set the lookup to run automatically. Use the Settings> Lookups > Automatic lookups page to edit or configure automatic lookups:

To edit an existing automatic lookup, select the lookup and modify the values in the fields displayed for that lookup.

Prerequisites

To add a new new lookup to run automatically:

  1. Select New in the Automatic lookups page:
  2. Select the Destination app.
  3. Select the Lookup table that you want use in your fields lookup.
    This is the name of the lookup definition that you define on the Lookup Definition page.
  4. In the Apply to menu, select a host, source, or source type value to apply the lookup and give it a name.
  5. Under Lookup input fields provide one or more pairs of input fields.
    The first field is the field in the lookup table that you want to match. The second field is a field from your events that should match the lookup table field. For example, you could have an ip_address field in your events that matches an ip field in the lookup table. So you would enter ip = ip_address in the automatic lookup definition.
  6. Under Lookup output fields provide one or more pairs of output fields.
    The first field is the corresponding field that you want to output to events. The second field is the name that the output field should have in your events. For example the lookup table may have a field named country that you may want to output to your events as ip_city.
  7. You can also select Overwrite field values to overwrite the field values each time the lookup runs. Note: This is equivalent to configuring your fields lookup in props.conf.

Example of HTTP status lookup

This examples walks through defining a static lookup that adds two informational fields, status_description and status_type, into your Web access events. This lets you search for the events you want when you might not know the specific error code. For example, instead of searching for all the server error codes, you can use status="Server Error".

Upload the lookup table to Splunk Enterprise

Prerequisities

Here's a sampling of the file:

status,status_description,status_type
100,Continue,Informational
101,Switching Protocols,Informational
200,OK,Successful
201,Created,Successful
202,Accepted,Successful
203,Non-Authoritative Information,Successful
...

Steps

  1. Go back to the Search app, then select Settings > Lookups.
    Manager from search b.png
  2. In the Lookups page, select Add new for Lookup table files. Add new lookup file b.png
  3. In the Add new page,
    • Select search for the destination app.
    • Browse for the CSV file that you downloaded earlier.
    • Name the lookup table http_status.
    • Click Save.
    Upload http status 4.2 b.png
    After Splunk Enterprise saves the file, it takes you to the following view: Upload lookup table file b.png
    Now, let's go back to the Settings > Lookups view. To do this, click on the Lookups link in the page's breadcrumb. You can always use this to navigate back to a previous view.
    Back to manager lookups b.png

Define the lookup

Prerequisites

Steps

  1. From Settings > Lookups, select Add new for Lookup definitions. In the Add new page: Manager add new lookup definition b.png
  2. Select search for the Destination app.
  3. Name your lookup definition http_status.
  4. Select File-based under Type.
  5. Click Save. After Splunk Enterprise saves your lookup definition, it takes you to the following page: Lookup def saved-b.png
    Notice there are some actions you can take on your lookup definition. Permissions lets you change the accessibility of the lookup table. You can Disable, Clone, and Move the lookup definition to a different app. Or, you can Delete the definition. Once you define the lookup, you can use the lookup command to invoke it in a search or you can configure the lookup to run automatically.

Set the lookup to run automatically

Prerequisites

Steps

  1. Return to the Settings > Lookups view and select Add new for Automatic lookups. In the Add new page: Add new automatic lookup b.png
  2. Select search for the Destination app.
  3. Name the lookup http_status.
  4. Select http_status from the Lookup table drop down.
  5. Apply the lookup to the sourcetype named access_combined.
    Apply lookup to field b.png
  6. Lookup input fields are the fields in our events that you want to match with the lookup table. Here, both are named status (the CSV column name goes on the left and the field that you want to match goes on the right): Lookup input fields b.png
  7. Lookup output fields are the fields from the lookup table that you want to add to your events: status_description and status_type. The CSV column name goes on the left and the field that you want to match goes on the right. Lookup output fields b.png
  8. Click Save.
    Auto lookup saved 4.2 b.png
PREVIOUS
About lookups and workflow actions
  NEXT
Introduction to lookup configuration

This documentation applies to the following versions of Splunk® Enterprise: 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters