Use field lookups to add information to your events

Use field lookups to add information to your events

Splunk's lookup feature lets you reference fields in an external CSV file that match fields in your event data. Using this match, you can enrich your event data by adding more searchable fields to them. You can base your field lookups on any field, including a temporal field, or on the output of a Python script.

This topic discusses how to use the Lookups manager page, located in Splunk Web, at Manager > Lookups, to:

• List existing lookup tables or upload a new file.
• Edit existing lookup definitions or define a new file-based or external lookup.
• Edit existing automatic lookups or configure a new lookup to run automatically.

For more details about lookups, see "Add fields from external data sources" in the Knowledge Manager manual.

List existing lookup tables or upload a new file

View existing lookup table files in Manager > Lookups > Lookup table files, or click "New" to upload more CSV files to use in your definitions for file-based lookups.

To upload new files:

1. Select a Destination app.

This tells Splunk to save your lookup table file in the app's directory: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.

2. Give your lookup table file a Name.

This will be the name you use to refer to the file in a lookup definition.

3. Browse for the CSV file to upload.

4. Click Save.

Edit existing lookup definitions or define a new file-based or external lookup

Use the Manager > Lookups > Lookup definitions page to define the lookup table or edit existing lookup definitions. You can specify the type of lookup (file-based or external) and whether or not it is time-based. Once you've defined the lookup table, you can invoke the lookup in a search (using the lookup command) or you can configure the lookup to occur automatically.

Note: This is equivalent to defining your lookup in transforms.conf.

Configure a time-based lookup

File-based and external lookups can also be time-based (or temporal), if the field matching depends on time information (a field in the lookup table that represents the timestamp).

To Configure a time-based lookup, specify the Name of the time field. You can also specify a strptime format for this time information and offsets for the time matching.

Include advanced options

Under Advanced options, you can specify:

• The minimum number of matches for each input lookup value.
• The maximum number of matches for each input lookup value.
• A default value to output if fewer than the minimum number of matches are present for a given input.

Edit existing automatic lookups or configure a new lookup to run automatically

Instead of invoking the lookup command when you want to apply a fields lookup to your events, you can set the lookup to run automatically. Use the Manager > Lookups > Automatic lookups page to edit or configure automatic lookups:

1. Select the lookup table file that you want use in your fields lookup.

2. Select a host, source, or sourcetype value to apply the lookup.

3. Under lookup input fields, list one or more pairs of lookup field name and local field name.

4. Under lookup output fields, list one or more pairs of lookup field name and local field name.

5. You can also choose to overwrite the field values each time the lookup runs.

Note: This is equivalent to configuring your fields lookup in props.conf.

Example of HTTP status lookup

This examples walks through defining a static lookup that adds two informational fields, status_description and status_type, into your Web access events. This lets you search for the events you want when you might not know the specific error code. For example, instead of searching for all the server error codes, you can use status="Server Error".

Upload the lookup table to Splunk

1. Download the http_status.csv file:

http_status.csv

Here's a sampling of the file:

status,status_description,status_type
100,Continue,Informational
101,Switching Protocols,Informational
200,OK,Successful
201,Created,Successful
202,Accepted,Successful
203,Non-Authoritative Information,Successful
...

2. Go back to the Search app, and select Manager from the navigation menu on the upper right.

3. In the Manager > Lookups view, select Add new for Lookup table files.

4. In Manager > Lookups > Lookup table files > Add new,

• Select search for the destination app.
• Browse for the CSV file that you downloaded earlier.
• Name the lookup table http_status.
• Click Save.

After Splunk saves the file, it takes you to the following view:

Now, let's go back to the Manager > Lookups view. To do this, click on the Lookups link in the page's breadcrumb. You can always use this to navigate back to a previous view.

Define the lookup

1. From Manager > Lookups, select Add new for Lookup definitions.

In the Manager > Lookups > Lookup definitions > Add new view:

2. Select search for the Destination app.

3. Name your lookup definition http_status.

4. Select File-based under Type.

5. Click Save.

After Splunk saves your lookup definition, it takes you to the following view:

Notice there are some actions you can take on your lookup definition. Permissions lets you change the accessibility of the lookup table. You can Disable, Clone, and Move the lookup definition to a different app. Or, you can Delete the definition.

Once you define the lookup, you can use the lookup command to invoke it in a search or you can configure the lookup to run automatically.

Set the lookup to run automatically

1. Return to the Manager > Lookups view and select Add new for Automatic lookups.

In the Manager > Lookups > Automatic lookups view:

2. Select search for the Destination app.

3. Name the lookup http_status.

4. Select http_status from the Lookup table drop down.

5. Apply the lookup to the sourcetype named access_combined.

6. Lookup input fields are the fields in our events that you want to match with the lookup table. Here, both are named status (the CSV column name goes on the left and the field that you want to match goes on the right):

7. Lookup output fields are the fields from the lookup table that you want to add to your events: status_description and status_type. The CSV column name goes on the left and the field that you want to match goes on the right.

8. Click Save.

• Was this topic useful?
• Post a comment