Related Answers
Download topic as PDF
outputlookup
Description
Writes search results to a static lookup table or KV store collection that you specify.
Syntax
| outputlookup [append=<bool>] [create_empty=<bool>] [max=<int>] [key_field=<field_name>] [createinapp=<bool>] (<filename> | <tablename>)
Required arguments
- <filename>
- Syntax: <string>
- Description: The name of the lookup file. The file must end with
.csvor.csv.gz.
- <tablename>
- Syntax: <string>
- Description: The name of the lookup table as specified by a stanza name in
transforms.conf. The lookup table can be configured for any lookup type (CSV, external, or KV store).
Optional arguments
- append
- Syntax: append=<bool>
- Description: If set to true,
outputlookupattempts to append search results to an existing.csvfile or KV store collection. Otherwise it creates a file. If there is an existing .csv fileoutputlookuponly writes the fields that are present in the previously existing.csvfile. Anoutputlookupsearch run withappend=truemay result in a situation where the lookup table or collection is only partially updated at some times. This means that a subsequentlookuporinputlookupsearch on that collection may return stale data along with new data.outputlookupcannot append to.gzfiles. - Default: false
- create_empty
- Syntax: create_empty=<bool>
- Description: If set to
trueand there are no results, creates a 0-length file. When set tofalse, no file is created. If the file previously existed, the file is deleted. - Default: true
- createinapp
- Syntax: createinapp=<bool>
- Description: If set to false or if there is no current application context, then create the file in the system lookups directory.
- Default: true
- key_field
- Syntax: key_field=<field_name>
- Description: For KV store-based lookups, uses the specified field name as the key to a value and replaces that value. An
outputlookupsearch using thekey_fieldargument might result in a situation where the lookup table or collection is only partially updated. A subsequentlookuporinputlookupsearch on that collection might return stale data along with new data. A partial update only occurs with concurrent searches, one with theoutputlookupcommand and a search with theinputlookupcommand. It is possible that theinputlookupoccurs when theoutputlookupis still updating some of the records.
- max
- Syntax: max=<int>
- Description: The number of rows to output.
- Default: no limit
Usage
The lookup table must be a CSV or GZ file, or a table name specified with a lookup table configuration in transforms.conf. The lookup table can refer to a KV store collection or a CSV lookup. The outputlookup command cannot be used with external lookups.
For CSV-based lookups, if the lookup file does not exist, it is created in the lookups directory of the current application. If the lookup file already exists, it is overwritten with the results of the outputlookup command. If the createinapp option is set to false or if there is no current application context, then the file is created in the system lookups directory.
For more information about lookup table configuration, see "Configure CSV and external lookups" and "Configure KV store lookups" in the Knowledge Manager Manual.
For more information about App Key Value Store collections, see "About KV store" in the Admin Manual.
Multivalued fields
When you output to a static lookup table, the outputlookup command merges values in a multivalued field into single space-delimited value. This does not apply to a KV store collection.
Examples
Example 1: Write to usertogroup lookup table as specified in transforms.conf.
| outputlookup usertogroup
Example 2: Write to users.csv lookup file under $SPLUNK_HOME/etc/system/lookups or $SPLUNK_HOME/etc/apps/*/lookups.
| outputlookup users.csv
Example 3: Write food inspection events for Shalimar Restaurant to a KV store collection called kvstorecoll. This collection is referenced in a lookup table called kvstorecoll_lookup.
index=sf_food_health sourcetype=sf_food_inspections name="SHALIMAR RESTAURANT" | outputlookup kvstorecoll_lookup
Example 4: Write the contents of a CSV file to the KV store collection kvstorecoll using the lookup table kvstorecoll_lookup. This requires usage of both inputlookup and outputlookup.
| inputlookup customers.csv | outputlookup kvstorecoll_lookup
Example 5: Update field values for a single KV store collection record. This requires usage of inputlookup, outputlookup, and eval. The record is indicated by the value of its internal key ID (the _key field) and is updated with a new customer name and customer city. The record belongs to the KV store collection kvstorecoll, which is accessed through the lookup table kvstorecoll_lookup.
| inputlookup kvstorecoll_lookup | search _key=544948df3ec32d7a4c1d9755 | eval CustName="Marge Simpson" | eval CustCity="Springfield" | outputlookup kvstorecoll_lookup append=True key_field=_key
To learn how to obtain the internal key ID values of the records in a KV store collection, see Example 5 for the inputlookup command.
See also
inputlookup, lookup, inputcsv, outputcsv
Answers
Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has using the outputlookup command.
|
PREVIOUS outputcsv |
NEXT outputtext |
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11
Comments
My last comment (obviously) applies to the Outputcsv documentation, too. Whatever is fixed here, should be copied and pasted there, also.
The handling of `multi-valued fields` is undocumented and is very important; outputcsv /outputlookup both call 'nomv' on all fields before writing out the rows (merging all multi-valued fields into single-space-delimited single values).
Notice the difference (change to field 'children') between the results of these 2 searches (which most would expect to be the same):
|noop|stats count AS name|eval name="Gregg"|eval spouse="Cindy"|eval children="Lauren Megan Noah"|makemv children
|streamstats count AS serial|eval mv_count=mvcount(children)|table serial name spouse children mv_count
|outputcsv eraseme.csv
|inputcsv eraseme.csv|streamstats count AS serial|eval mv_count=mvcount(children)|table serial name spouse children mv_count
It would be nice to have an option to these commands to choose between 'nomv' and 'mvexpand'.
Richgalloway - Thanks for noticing the issue with Example 5 and your question about the key_field argument.
I fixed the example.
For your question about the description of key_field, yes, this only happens with concurrent queries, one with outputlookup and one with inputlookup. It could happen that the inputlookup would happen while the outputlookup was still updating some of the records. I'll update the description with this information.
The description of the key_field argument says "An outputlookup search using the key_field argument might result in a situation where the lookup table or collection is only partially updated. This means that a subsequent lookup or inputlookup search on that collection might return stale data along with new data." When would a partial update occur? Is this only a concern when outputlookup and inputlookup are simultaneous in separate searches?
Example 5 does not use the key_field argument as would seem to be required to perform an update. Does key_field have a default value that is not documented?
Woodcock - Thanks for noticing this. I have updated the sentence.
This statement is incorrect (a copy/paste error from "inputlookup", probably, where it is true):
The outputlookup command is a generating commandand should be the first command in the search. Generating commands use a leading pipe character.
Woodcock
You are correct, we don't preserve the internal multivalue encoding scheme for outputcsv or outputlookup. The mv fields are flattened. I will forward your request to choose between 'nomv' and 'mvexpand' to our development team.