Splunk® Enterprise

Search Reference

Splunk Enterprise version 8.0 is no longer supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

lookup

Description

Use the lookup command to invoke field value lookups.

For information about the types of lookups you can define, see About lookups in the Knowledge Manager Manual.

Syntax

The required syntax is in bold.

lookup
[local=<bool>]
[update=<bool>]
<lookup-table-name>
( <lookup-field> [AS <event-field>] )...
[ OUTPUT | OUTPUTNEW (<lookup-destfield> [AS <event-destfield>] )... ]

Note: The lookup command can accept multiple lookup and event fields and destfields. For example:

...| lookup <lookup-table-name> <lookup-field1> AS <event-field1>, <lookup-field2> AS <event-field2> OUTPUTNEW <lookup-destfield1> AS <event-destfield1>, <lookup-destfield2> AS <event-destfield2>

Required arguments

<lookup-table-name>
Syntax: <string>
Description: Can be either the name of a CSV file that you want to use as the lookup, or the name of a stanza in the transforms.conf file that specifies the location of the lookup table file.

Optional arguments

local
Syntax: local=<bool>
Description: If local=true, forces the lookup to run on the search head and not on any remote peers.
Default: false
update
Syntax: update=<bool>
Description: If the lookup table is modified on disk while the search is running, real-time searches do not automatically reflect the update. To do this, specify update=true. This does not apply to searches that are not real-time searches. This implies that local=true.
Default: false
<lookup-field>
Syntax: <string>
Description: Refers to a field in the lookup table to match against the events. You can specify multiple <lookup-field> values.
<event-field>
Syntax: <string>
Description: Refers to a field the events from which to acquire the value to match in the lookup table. You can specify multiple <event-field> values.
Default: The value of the <lookup-field>.
<lookup-destfield>
Syntax: <string>
Description: Refers to a field in the lookup table to be copied into the events. You can specify multiple <lookup-destfield> values.
<event-destfield>
Syntax: <string>
Description: A field in the events. You can specify multiple <event-destfield> values.
Default: The value of the <lookup-destfield> argument.

Usage

The lookup command is a distributable streaming command when local=false, which is the default setting. See Command types.

When using the lookup command, if an OUTPUT or OUTPUTNEW clause is not specified, all of the fields in the lookup table that are not the match fields are used as output fields. If the OUTPUT clause is specified, the output lookup fields overwrite existing fields. If the OUTPUTNEW clause is specified, the lookup is not performed for events in which the output fields already exist.

Avoid lookup reference cycles

When you set up the OUTPUT or OUTPUTNEW clause for your lookup, avoid accidentally creating lookup reference cycles, where you intentionally or accidentally reuse the same field names among the match fields and the output fields of a lookup search.

For example, if you run a lookup search where type is both the match field and the output field, you are creating a lookup reference cycle. You can accidentally create a lookup reference cycle when you fail to specify an OUTPUT or OUTPUTNEW clause for lookup.

For more information about lookup reference cycles see Define an automatic lookup in Splunk Web in the Knowledge Manager Manual.

Optimizing your lookup search

If you are using the lookup command in the same pipeline as a transforming command, and it is possible to retain the field you will lookup on after the transforming command, do the lookup after the transforming command. For example, run:

sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description

and not:

sourcetype=access_* | lookup status_desc status OUTPUT description | stats count by description

The lookup in the first search is faster because it only needs to match the results of the stats command and not all the Web access events.

Basic example

1. Lookup users and return the corresponding group the user belongs to

Suppose you have a lookup table specified in a stanza named usertogroup in the transforms.conf file. This lookup table contains (at least) two fields, user and group. Your events contain a field called local_user. For each event, the following search checks to see if the value in the field local_user has a corresponding value in the user field in the lookup table. For any entries that match, the value of the group field in the lookup table is written to the field user_group in the event.

... | lookup usertogroup user as local_user OUTPUT group as user_group

Extended example

2. Lookup price and vendor information and return the count for each product sold by a vendor

This example uses the tutorialdata.zip file from the Search Tutorial. You can download this file and follow the instructions to upload the tutorial data into your Splunk deployment. Additionally, this example uses the prices.csv and the vendors.csv files. To follow along with this example in your Splunk deployment, download these CSV files and complete the steps in the Use field lookups section of the Search Tutorial for both the prices.csv and the vendors.csv files. When you create the lookup definition for the vendors.csv file, name the lookup vendors_lookup. You can skip the step in the tutorial that makes the lookups automatic.

This example calculates the count of each product sold by each vendor.

The prices.csv file contains the product names, price, and code. For example:

productId product_name price sale_price Code
DB-SG-G01 Mediocre Kingdoms 24.99 19.99 A
DC-SG-G02 Dream Crusher 39.99 24.99 B
FS-SG-G03 Final Sequel 24.99 16.99 C
WC-SH-G04 World of Cheese 24.99 19.99 D

The vendors.csv file contains vendor information, such as vendor name, city, and ID. For example:

Vendor VendorCity VendorID VendorLatitude VendorLongitude Vendor StateProvince Vendor Country Weight
Anchorage Gaming Anchorage 1001 61.17440033 -149.9960022 Alaska United States 3
Games of Salt Lake Salt Lake City 1002 40.78839874 -111.9779968 Utah United States 3
New Jack Games New York 1003 40.63980103 -73.77890015 New York United States 4
Seals Gaming San Francisco 1004 37.61899948 -122.375 California United States 5

The search will query the vendor_sales.log file, which is part of the tutorialdata.zip file. The vendor_sales.log file contains the VendorID, Code, and AcctID fields. For example:

Entries in the vendor_sales.log file
[13/Mar/2018:18:24:02] VendorID=5036 Code=B AcctID=6024298300471575
[13/Mar/2018:18:23:46] VendorID=7026 Code=C AcctID=8702194102896748
[13/Mar/2018:18:23:31] VendorID=1043 Code=B AcctID=2063718909897951
[13/Mar/2018:18:22:59] VendorID=1243 Code=F AcctID=8768831614147676

The following search calculates the count of each product sold by each vendor and uses the time range All time.

sourcetype=vendor_* | stats count by Code VendorID | lookup prices_lookup Code OUTPUTNEW product_name

  • The stats command calculates the count by Code and VendorID.
  • The lookup command uses the prices_lookup to match the Code field in each event and return the product names.

The search results are displayed on displayed on the Statistics tab.

This image shows the results of the search in the Statistics tab. There are more than 30,000 events returned. There are four columns in the output. The first column contains the Code values. The second column contains the Vendor IDs. The third column contains the count by Vendor ID. The last column contains the names of the products.


You can extend the search to display more information about the vendor by using the vendors_lookup.

Use the table command to return only the fields that you need. In this example you want the product_name, VendorID, and count fields. Use the vendors_lookup file to output all the fields in the vendors.csv file that match the VendorID in each event.

sourcetype=vendor_* | stats count by Code VendorID | lookup prices_lookup Code OUTPUTNEW product_name | table product_name VendorID count | lookup vendors_lookup VendorID

The revised search results are displayed on the Statistics tab.

This image shows the results of the search in the Statistics tab. The search returns nine columns: product name, vendor ID, vendor, vendor city, vendor country, vendor latitude, vendor longitude, vendor state or province, and weight.


To expand the search to display the results on a map, see the geostats command.

See also

Commands
appendcols
inputlookup
outputlookup
Related information
About lookups in the Knowledge Manager Manual
Last modified on 01 December, 2021
 

This documentation applies to the following versions of Splunk® Enterprise: 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters