Splunk® Enterprise

Search Reference

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

outputlookup

Description

Writes search results to a static lookup table or KV store collection that you specify.

Syntax

| outputlookup [append=<bool>] [create_empty=<bool>] [max=<int>] [key_field=<field_name>] [createinapp=<bool>] (<filename> | <tablename>)

Required arguments

<filename>
Syntax: <string>
Description: The name of the lookup file. The file must end with .csv or .csv.gz.
<tablename>
Syntax: <string>
Description: The name of the lookup table as specified by a stanza name in transforms.conf. The lookup table can be configured for any lookup type (CSV, external, or KV store).

Optional arguments

append
Syntax: append=<bool>
Description: If set to true, outputlookup attempts to append search results to an existing .csv file or KV store collection. Otherwise it creates a file. If there is an existing .csv file outputlookup only writes the fields that are present in the previously existing .csv file. An outputlookup search run with append=true may result in a situation where the lookup table or collection is only partially updated at some times. This means that a subsequent lookup or inputlookup search on that collection may return stale data along with new data. outputlookup cannot append to .gz files.
Default: false
create_empty
Syntax: create_empty=<bool>
Description: If set to true and there are no results, creates a 0-length file. When set to false, no file is created. If the file previously existed, the file is deleted.
Default: true
createinapp
Syntax: createinapp=<bool>
Description: If set to false or if there is no current application context, then create the file in the system lookups directory.
Default: true
key_field
Syntax: key_field=<field_name>
Description: For KV store-based lookups, uses the specified field name as the key to a value and replaces that value. An outputlookup search using the key_field argument might result in a situation where the lookup table or collection is only partially updated. A subsequent lookup or inputlookup search on that collection might return stale data along with new data. A partial update only occurs with concurrent searches, one with the outputlookup command and a search with the inputlookup command. It is possible that the inputlookup occurs when the outputlookup is still updating some of the records.
max
Syntax: max=<int>
Description: The number of rows to output.
Default: no limit

Usage

The lookup table must be a CSV or GZ file, or a table name specified with a lookup table configuration in transforms.conf. The lookup table can refer to a KV store collection or a CSV lookup. The outputlookup command cannot be used with external lookups.

For CSV-based lookups, if the lookup file does not exist, it is created in the lookups directory of the current application. If the lookup file already exists, it is overwritten with the results of the outputlookup command. If the createinapp option is set to false or if there is no current application context, then the file is created in the system lookups directory.

For more information about lookup table configuration, see "Configure CSV and external lookups" and "Configure KV store lookups" in the Knowledge Manager Manual.

For more information about App Key Value Store collections, see "About KV store" in the Admin Manual.

Multivalued fields

When you output to a static lookup table, the outputlookup command merges values in a multivalued field into single space-delimited value. This does not apply to a KV store collection.

Examples

Example 1: Write to usertogroup lookup table as specified in transforms.conf.

| outputlookup usertogroup

Example 2: Write to users.csv lookup file under $SPLUNK_HOME/etc/system/lookups or $SPLUNK_HOME/etc/apps/*/lookups.

| outputlookup users.csv

Example 3: Write food inspection events for Shalimar Restaurant to a KV store collection called kvstorecoll. This collection is referenced in a lookup table called kvstorecoll_lookup.

index=sf_food_health sourcetype=sf_food_inspections name="SHALIMAR RESTAURANT" | outputlookup kvstorecoll_lookup

Example 4: Write the contents of a CSV file to the KV store collection kvstorecoll using the lookup table kvstorecoll_lookup. This requires usage of both inputlookup and outputlookup.

| inputlookup customers.csv | outputlookup kvstorecoll_lookup

Example 5: Update field values for a single KV store collection record. This requires usage of inputlookup, outputlookup, and eval. The record is indicated by the value of its internal key ID (the _key field) and is updated with a new customer name and customer city. The record belongs to the KV store collection kvstorecoll, which is accessed through the lookup table kvstorecoll_lookup.

| inputlookup kvstorecoll_lookup | search _key=544948df3ec32d7a4c1d9755 | eval CustName="Marge Simpson" | eval CustCity="Springfield" | outputlookup kvstorecoll_lookup append=True key_field=_key

To learn how to obtain the internal key ID values of the records in a KV store collection, see Example 5 for the inputlookup command.

See also

inputlookup, lookup, inputcsv, outputcsv

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has using the outputlookup command.

PREVIOUS
outputcsv
  NEXT
outputtext

This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11


Comments

Woodcock
You are correct, we don't preserve the internal multivalue encoding scheme for outputcsv or outputlookup. The mv fields are flattened. I will forward your request to choose between 'nomv' and 'mvexpand' to our development team.

Lstewart splunk, Splunker
April 12, 2016

My last comment (obviously) applies to the Outputcsv documentation, too. Whatever is fixed here, should be copied and pasted there, also.

Woodcock
April 10, 2016

The handling of `multi-valued fields` is undocumented and is very important; outputcsv /outputlookup both call 'nomv' on all fields before writing out the rows (merging all multi-valued fields into single-space-delimited single values).

Notice the difference (change to field 'children') between the results of these 2 searches (which most would expect to be the same):

|noop|stats count AS name|eval name="Gregg"|eval spouse="Cindy"|eval children="Lauren Megan Noah"|makemv children
|streamstats count AS serial|eval mv_count=mvcount(children)|table serial name spouse children mv_count
|outputcsv eraseme.csv

|inputcsv eraseme.csv|streamstats count AS serial|eval mv_count=mvcount(children)|table serial name spouse children mv_count

It would be nice to have an option to these commands to choose between 'nomv' and 'mvexpand'.

Woodcock
April 10, 2016

Richgalloway - Thanks for noticing the issue with Example 5 and your question about the key_field argument.

I fixed the example.

For your question about the description of key_field, yes, this only happens with concurrent queries, one with outputlookup and one with inputlookup. It could happen that the inputlookup would happen while the outputlookup was still updating some of the records. I'll update the description with this information.

Lstewart splunk, Splunker
March 25, 2016

The description of the key_field argument says "An outputlookup search using the key_field argument might result in a situation where the lookup table or collection is only partially updated. This means that a subsequent lookup or inputlookup search on that collection might return stale data along with new data." When would a partial update occur? Is this only a concern when outputlookup and inputlookup are simultaneous in separate searches?

Richgalloway
March 24, 2016

Example 5 does not use the key_field argument as would seem to be required to perform an update. Does key_field have a default value that is not documented?

Richgalloway
March 24, 2016

Woodcock - Thanks for noticing this. I have updated the sentence.

Lstewart splunk, Splunker
February 8, 2016

This statement is incorrect (a copy/paste error from "inputlookup", probably, where it is true):

The outputlookup command is a generating commandand should be the first command in the search. Generating commands use a leading pipe character.

Woodcock
February 8, 2016

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters