Splunk® Enterprise

Knowledge Manager Manual

Acrobat logo Download manual as PDF


Splunk Enterprise version 7.0 is no longer supported as of October 23, 2019. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Define a CSV lookup in Splunk Web

CSV lookups are file-based lookups that match field values from your events to field values in the static table represented by a CSV file. They output corresponding field values from the table to your events. They are also referred to as static lookups.

CSV lookups are best for small sets of data. The general workflow for creating a CSV lookup in Splunk Web is to upload a file, share the lookup table file, and then create the lookup definition from the lookup table file. CSV inline lookup table files, and inline lookup definitions that use CSV files, are both dataset types. See Dataset types and usage.

About the CSV files

There are some restrictions to the files that can be used for CSV lookups.

  • The table in the CSV file should have at least two columns. One column represents a field with a set of values that includes values belonging to a field in your events. The column does not have to have the same name as the event field. Any column can have multiple instances of the same value, which is a multivalued field.
  • The characters in the CSV file must be plain ASCII text and valid UTF-8 characters. Non-UTF-8 characters are not supported.
  • CSV files cannot have "\r" line endings (OSX 9 or earlier)
  • CSV files cannot have header rows that exceed 4096 characters.

Upload the lookup table file

To use a lookup table file, you must upload the file to your Splunk platform.

Prerequisites

  • See lookup for an example of how to define a CSV lookup.
  • An available .csv or .gz table file.

Steps

  1. Select Settings > Lookups to go to the Lookups manager page.
  2. In the Actions column, click Add new next to Lookup table files.
  3. Select a Destination app from the list.
    Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
  4. Click Choose File to look for the CSV file to upload.
  5. Enter the destination filename. This is the name the lookup table file will have on the Splunk server. If you are uploading a gzipped CSV file, enter a filename ending in ".gz". If you are uploading a plaintext CSV file, use a filename ending in ".csv".
  6. Click Save.

Share a lookup table file with apps

After you upload the lookup file, tell the Splunk software which applications can use this file. The default app is Launcher.

  1. Select Settings > Lookups.
  2. From the Lookup manager, click Lookup table files.
  3. Click Permissions in the Sharing column of the lookup you want to share.
  4. In the Permissions dialog box, under Object should appear in, select All apps to share globally. If you want the lookup to be specific to this app only, select This app only. You can also keep your lookup private by selecting Keep private.
  5. Click Save.

Create a CSV lookup definition

You must create a lookup definition from the lookup table file.

Prerequisites
In order to create the lookup definition, share the lookup table file so that Splunk software can see it.

Review

Steps

  1. Select Settings > Lookups.
  2. Click Lookup definitions.
  3. Click New.
  4. Select a Destination app from the drop-down list. Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
  5. Give your lookup definition a unique Name.
  6. Select File-based as the lookup Type.
  7. Select the Lookup file from the drop-down list. For a CSV lookup, the file extension must be .csv.
  8. (Optional) If the CSV file contains time fields, make the CSV lookup time-bounded by selecting the Configure time-based lookup check box.
    Time-based options Description
    Name of time field The name of the field in the lookup table that represents the timestamp. This defaults to 0.
    Time format The strptime format of the timestamp field. You can include subseconds but the Splunk platform will ignore them. This defaults to %s.%Q or seconds from unix epoch in UTC and optional milliseconds.
    Minimum offset The minimum time (in seconds) that the event timestamp can be later than the lookup entry timestamp for a match to occur. This defaults to 0.
    Maximum offset The maximum time (in seconds) that the event timestamp can be later than the lookup entry time for a match to occur. This defaults to 2000000000.
  9. (Optional) To define advanced options for your lookup, select the Advanced options check box.
    Advanced options Description
    Minimum matches The minimum number of matches for each input lookup value. Defaults to 0.
    Maximum matches Enter a number from 1-1000 to specify the maximum number of matches for each lookup value. If time-based, the default value is 1; otherwise, the default value is 1000.
    Default matches When fewer than the minimum number of matches are present for any given input, the Splunk software provides this value one or more times until the minimum is reached.
    Case sensitive match If the check box is selected, case-sensitive matching will be performed for all fields in a lookup table. The default value is true.
    Batch index query Select this check box if you are using a large lookup file that may affect performance.
    Match type A comma and space-delimited list of <match_type>(<field_name>) specification to allow for non-exact matching. The available match_type values are WILDCARD, CIDR, and EXACT. EXACT is the default. Specify the fields that use WILDCARD or CIDR in this list.
    Filter lookup Filter results from the lookup table before returning data. Create this filter like you would a typical search query using Boolean expressions and/or comparison operators.

    For CSV lookups, filtering is done in memory.

  10. Click Save.

Your lookup is defined as a file-based CSV lookup and appears in the list of lookup definitions.

Share the lookup definition with apps

After you create the lookup definition, specify in which apps you want to use the definition.

  1. Select Settings > Lookups.
  2. Click Lookup definitions.
  3. In the Lookup definitions list, click Permissions in the Sharing column of the lookup definition you want to share.
  4. In the Permissions dialog box, under Object should appear in, select All apps to share globally. If you want the lookup to be specific to this app only, select This app only. You can also keep your lookup private by selecting Keep private.
  5. Click Save.

Permissions for lookup table files must be at the same level or higher than those of the lookup definitions that use those files.

You can use this field lookup to add information from the lookup table file to your events. You can use the field lookup with the lookup command in a search string. Or, you can set the field lookup to run automatically. For information on creating an automatic lookup, see Create a new lookup to run automatically.

Make the lookup automatic

Instead of using the lookup command in your search when you want to apply a field lookup to your events, you can set the lookup to run automatically. See Define an automatic lookup for more information.

Configure a CSV lookup with .conf files

CSV lookups can also be configured using .conf files. See Configure CSV lookups.

Last modified on 26 November, 2018
PREVIOUS
About lookups
  NEXT
Define an external lookup in Splunk Web

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters