Splunk® Enterprise

Search Reference

Download manual as PDF

Download topic as PDF

collect

Description

Adds the results of a search to a summary index that you specify. You must create the summary index before you invoke the collect command.

Syntax

collect index=<string> [<arg-options>...]

Required arguments

index
Syntax: index=<string>
Description: Name of the summary index where the events are added. The index must exist before the events are added. The index is not created automatically.

Optional arguments

arg-options
Syntax: addtime=<bool> | file=<string> | spool=<bool> | marker=<string> | testmode=<bool> | run_in_preview=<bool> | host=<string> | source=<string> | sourcetype=<string>
Description: Optional arguments for the collect command. See the arg-options section for the descriptions for each option.

arg-options

addtime
Syntax: addtime=<bool>
Description: Use this option to specify whether to prefix a time field on to each event. Some commands return results that do not have a _raw field, such as the stats, chart, timechart commands. If you specify addtime=false, the Splunk software uses its generic date detection against fields in whatever order they happen to be in the summary rows. If you specify addtime=true, the Splunk software uses the search time range info_min_time. This time range is added by the sistats) command or _time. Splunk software adds the time field based on the first field that it finds: info_min_time, _time, or now().
Default: true
file
Syntax: file=<string>
Description: The file name where you want the events to be written. You can use a timestamp or a random number for the file name by specifying either file=$timestamp$ or file=$random$.
Usage: ".stash" needs to be added at the end of the file name when used with "index=". Otherwise, the data is added to the main index.
Default: <random-number>_events.stash
host
Syntax: host=<string>
Description: The name of the host that you want to specify for the events.
marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Each key-value pair must be separated by a comma and a space.
If the value contains spaces or commas, it must be escape quoted. For example if the key-value pair is search_name=vpn starts and stops, you must change it to search_name=\"vpn starts and stops\".
run_in_preview
Syntax: run_in_preview=<bool>
Description: Controls whether the collect command is enabled during preview generation. Generally, you do not want to insert preview results into the summary index, run-in-preview=false. In some cases, such as when a custom search command is used as part of the search, you might want to turn this on to ensure correct summary indexable previews are generated.
Default: false
spool
Syntax: spool=<bool>
Description: If set to true, the summary indexing file is written to the Splunk spool directory, where it is indexed automatically. If set to false, the file is written to the $SPLUNK_HOME/var/run/splunk directory. The file remains in this directory unless some form of further automation or administration is done. If you have Splunk Enterprise, you can use this command to troubleshoot summary indexing by dumping the output file to a location on disk where it will not be ingested as data.
Default: true
source
Syntax: source=<string>
Description: The name of the source that you want to specify for the events.
sourcetype
Syntax: sourcetype=<string>
Description: The name of the source type that you want to specify for the events. By specifying a sourcetype outside of stash, you will incur license usage.
Default: stash
testmode
Syntax: testmode=<bool>
Description: Toggle between testing and real mode. In testing mode the results are not written into the new index but the search results are modified to appear as they would if sent to the index.
Default: false

Usage

The events are written to a file whose name format is: random-num_events.stash, unless overwritten, in a directory that your Splunk deployment is monitoring. If the events contain a _raw field, then this field is saved. If the events do not have a _raw field, one is created by concatenating all the fields into a comma-separated list of key=value pairs.

The collect command also works with real-time searches that have a time range of All time.

Events without timestamps

If you apply the collect command to events that do not have timestamps, the command designates a time for all of the events using the earliest (or minimum) time of the search range. For example, if you use the collect command over the past four hours (range: -4h to +0h), the command assigns a timestamp that is four hours prior to the time that the search was launched. The timestamp is applied to all of the events without a timestamp.

If you use the collect command with a time range of All time and the events do not have timestamps, the current system time is used for the timestamps.

For more information on summary indexing of data without timestamps, see "Use summary indexing for increased reporting efficiency" in the Knowledge Manager Manual.

Moving events to a different index

You can use the collect command to move selected file content from one index to another index. Construct a search that returns the data you want to port, and pipe the results to the collect command. For example:

index=whatever host=whatever source=whatever whatever | collect index=foo

This search ports the data into the foo index. The sourcetype is changed to stash.

You can specify a sourcetype with the collect command. However, specifying a sourcetype counts against your license, as if you indexed the data again.

Examples

1. Put "download" events into an index named "download count"

eventtypetag="download" | collect index=downloadcount

2. Collect statistics on VPN connects and disconnects

You want to collect hourly statistics on VPN connects and disconnects by country.

index=mysummary | geoip REMOTE_IP | eval country_source=if(REMOTE_IP_country_code="US","domestic","foreign") | bin _time span=1h | stats count by _time,vpn_action,country_source | addinfo | collect index=mysummary marker="summary_type=vpn, summary_span=3600, summary_method=bin, search_name=\"vpn starts and stops\""

The addinfo command ensures that the search results contain fields that specify when the search was run to populate these particular index values.

See also

overlap, sichart, sirare, sistats, sitop, sitimechart, tscollect

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has using the collect command.

PREVIOUS
cofilter
  NEXT
concurrency

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.1.0, 7.0.4, 7.1.1


Comments

arg-options is missing the "timeformat" option. See https://answers.splunk.com/answers/110993/can-the-formatting-of-the-timestamp-added-by-the-collect-command-be-modified.html

My2ndhead
January 4, 2017

It appears the collect command supports setting the sourcetype per http://answers.splunk.com/answers/34946/move-some-content-source-from-one-index-to-another-index.html.<br /><br />| collect index=foo sourcetype=bar

Runals
October 13, 2014

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters