lookup
Description
Use the lookup
command to invoke field value lookups.
For information about the types of lookups you can define, see About lookups in the Knowledge Manager Manual.
Syntax
The required syntax is in bold.
- lookup
- [local=<bool>]
- [update=<bool>]
- <lookup-table-name>
- ( <lookup-field> [AS <event-field>] )...
- [ OUTPUT | OUTPUTNEW (<lookup-destfield> [AS <event-destfield>] )... ]
Note: The lookup command can accept multiple lookup and event fields
and destfields
. For example:
...| lookup <lookup-table-name> <lookup-field1> AS <event-field1>, <lookup-field2> AS <event-field2> OUTPUTNEW <lookup-destfield1> AS <event-destfield1>, <lookup-destfield2> AS <event-destfield2>
Required arguments
- <lookup-table-name>
- Syntax: <string>
- Description: Can be either the name of a CSV file that you want to use as the lookup, or the name of a stanza in the
transforms.conf
file that specifies the location of the lookup table file.
Optional arguments
- local
- Syntax: local=<bool>
- Description: If
local=true
, forces the lookup to run on the search head and not on any remote peers. - Default: false
- update
- Syntax: update=<bool>
- Description: If the lookup table is modified on disk while the search is running, real-time searches do not automatically reflect the update. To do this, specify
update=true
. This does not apply to searches that are not real-time searches. This implies that local=true. - Default: false
- <lookup-field>
- Syntax: <string>
- Description: Refers to a field in the lookup table to match against the events. You can specify multiple <lookup-field> values.
- <event-field>
- Syntax: <string>
- Description: Refers to a field the events from which to acquire the value to match in the lookup table. You can specify multiple <event-field> values.
- Default: The value of the <lookup-field>.
- <lookup-destfield>
- Syntax: <string>
- Description: Refers to a field in the lookup table to be copied into the events. You can specify multiple <lookup-destfield> values.
- <event-destfield>
- Syntax: <string>
- Description: A field in the events. You can specify multiple <event-destfield> values.
- Default: The value of the <lookup-destfield> argument.
Usage
The lookup
command is a distributable streaming command when local=false
, which is the default setting.
See Command types.
When using the lookup
command, if an OUTPUT or OUTPUTNEW clause is not specified, all of the fields in the lookup table that are not the match fields are used as output fields. If the OUTPUT clause is specified, the output lookup fields overwrite existing fields. If the OUTPUTNEW clause is specified, the lookup is not performed for events in which the output fields already exist.
Avoid lookup reference cycles
When you set up the OUTPUT or OUTPUTNEW clause for your lookup, avoid accidentally creating lookup reference cycles, where you intentionally or accidentally reuse the same field names among the match fields and the output fields of a lookup
search.
For example, if you run a lookup
search where type
is both the match field and the output field, you are creating a lookup reference cycle. You can accidentally create a lookup reference cycle when you fail to specify an OUTPUT or OUTPUTNEW clause for lookup
.
For more information about lookup reference cycles see Define an automatic lookup in Splunk Web in the Knowledge Manager Manual.
Optimizing your lookup search
If you are using the lookup
command in the same pipeline as a transforming command, and it is possible to retain the field you will lookup on after the transforming command, do the lookup after the transforming command. For example, run:
sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description
and not:
sourcetype=access_* | lookup status_desc status OUTPUT description | stats count by description
The lookup in the first search is faster because it only needs to match the results of the stats command and not all the Web access events.
Basic example
1. Lookup users and return the corresponding group the user belongs to
Suppose you have a lookup table specified in a stanza named usertogroup
in the transforms.conf
file. This lookup table contains (at least) two fields, user
and group
. Your events contain a field called local_user
. For each event, the following search checks to see if the value in the field local_user
has a corresponding value in the user
field in the lookup table. For any entries that match, the value of the group
field in the lookup table is written to the field user_group
in the event.
... | lookup usertogroup user as local_user OUTPUT group as user_group
Extended example
2. Lookup price and vendor information and return the count for each product sold by a vendor
This example uses the tutorialdata.zip file from the Search Tutorial. You can download this file and follow the instructions to upload the tutorial data into your Splunk deployment. Additionally, this example uses the prices.csv and the vendors.csv files. To follow along with this example in your Splunk deployment, download these CSV files and complete the steps in the Use field lookups section of the Search Tutorial for both the prices.csv and the vendors.csv files. When you create the lookup definition for the vendors.csv file, name the lookup vendors_lookup. You can skip the step in the tutorial that makes the lookups automatic.
|
This example calculates the count of each product sold by each vendor.
The prices.csv
file contains the product names, price, and code. For example:
productId | product_name | price | sale_price | Code |
---|---|---|---|---|
DB-SG-G01 | Mediocre Kingdoms | 24.99 | 19.99 | A |
DC-SG-G02 | Dream Crusher | 39.99 | 24.99 | B |
FS-SG-G03 | Final Sequel | 24.99 | 16.99 | C |
WC-SH-G04 | World of Cheese | 24.99 | 19.99 | D |
The vendors.csv
file contains vendor information, such as vendor name, city, and ID. For example:
Vendor | VendorCity | VendorID | VendorLatitude | VendorLongitude | Vendor StateProvince | Vendor Country | Weight |
---|---|---|---|---|---|---|---|
Anchorage Gaming | Anchorage | 1001 | 61.17440033 | -149.9960022 | Alaska | United States | 3 |
Games of Salt Lake | Salt Lake City | 1002 | 40.78839874 | -111.9779968 | Utah | United States | 3 |
New Jack Games | New York | 1003 | 40.63980103 | -73.77890015 | New York | United States | 4 |
Seals Gaming | San Francisco | 1004 | 37.61899948 | -122.375 | California | United States | 5 |
The search will query the vendor_sales.log
file, which is part of the tutorialdata.zip file. The vendor_sales.log
file contains the VendorID, Code, and AcctID fields. For example:
Entries in the vendor_sales.log file |
---|
[13/Mar/2018:18:24:02] VendorID=5036 Code=B AcctID=6024298300471575 |
[13/Mar/2018:18:23:46] VendorID=7026 Code=C AcctID=8702194102896748 |
[13/Mar/2018:18:23:31] VendorID=1043 Code=B AcctID=2063718909897951 |
[13/Mar/2018:18:22:59] VendorID=1243 Code=F AcctID=8768831614147676 |
The following search calculates the count of each product sold by each vendor and uses the time range All time
.
sourcetype=vendor_* | stats count by Code VendorID | lookup prices_lookup Code OUTPUTNEW product_name
- The
stats
command calculates thecount
by Code and VendorID. - The
lookup
command uses theprices_lookup
to match the Code field in each event and return the product names.
The search results are displayed on displayed on the Statistics tab.
You can extend the search to display more information about the vendor by using the vendors_lookup.
Use the table
command to return only the fields that you need. In this example you want the product_name
, VendorID
, and count
fields. Use the vendors_lookup
file to output all the fields in the vendors.csv
file that match the VendorID in each event.
sourcetype=vendor_* | stats count by Code VendorID | lookup prices_lookup Code OUTPUTNEW product_name | table product_name VendorID count | lookup vendors_lookup VendorID
The revised search results are displayed on the Statistics tab.
To expand the search to display the results on a map, see the geostats command.
See also
- Commands
- appendcols
- inputlookup
- outputlookup
- Related information
- About lookups in the Knowledge Manager Manual
localop | makecontinuous |
This documentation applies to the following versions of Splunk® Enterprise: 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10
Feedback submitted, thanks!