set

Description

Performs set operations on subsearches.

Syntax

The required syntax is in bold.

| set (union | diff | intersect) subsearch1 subsearch2

Required arguments

union | diff | intersect

Syntax: union | diff | intersect

Description: Performs two subsearches, then executes the specified set operation on the two sets of search results.

Operation	Description
union	Returns a set that combines the results generated by the two subsearches. Provides results that are common to both subsets only once.
diff	Returns a set that combines the results generated by the two subsearches and excludes the events common to both. Does not indicate which subsearch the results originated from.
intersect	Returns a set that contains results common to both subsearches.

subsearch

Syntax: "[" <string> "]"

Description: Specifies a subsearch. Subsearches must be enclosed in square brackets. For more information about subsearch syntax, see "About subsearches" in the Search Manual.

Usage

The set command is an event-generating command. See Command types.

Generating commands use a leading pipe character and should be the first command in a search.

Results

The set command considers results to be the same if all of fields that the results contain match. Some internal fields generated by the search, such as _serial, vary from search to search. You need to filter out some of the fields if you are using the set command with raw events, as opposed to transformed results such as those from a stats command. Typically in these cases, all fields are the same from search to search.

Output limitations

There is a limit on the quantity of results that come out of the invoked subsearches that the set command receives to operate on. If this limit is exceeded, the input result set to the diff command is silently truncated.

If you have Splunk Enterprise, you can adjust this limit by editing the limits.conf file and changing the maxout value in the [subsearch] stanza. If this value is altered, the default quantity of results coming from a variety of subsearch scenarios are altered. Note that very large values might cause extensive stalls during the 'parsing' phase of a search, which is when subsearches run. The default value for this limit is 10000.

Only users with file system access, such as system administrators, can edit the configuration files. Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make the changes in the local directory.

See How to edit a configuration file.

If you have Splunk Cloud Platform and want to edit a configuration file, file a Support ticket.

Result rows limitations

By default the set command attempts to traverse a maximum of 50000 items from each subsearch. If the number of input results from either search exceeds this limit, the set command silently ignores the remaining events. By default, the maxout setting for subsearches in limits.conf prevents the number of results from exceeding this limit.

This maximum is controlled by the maxresultrows setting in the [set] stanza in the limits.conf file. Increasing this limit can result in more memory usage.

Only users with file system access, such as system administrators, can edit the configuration files. Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make the changes in the local directory.

See How to edit a configuration file.

If you have Splunk Cloud Platform and want to edit a configuration file, file a Support ticket.

Examples

Example 1:

Return values of "URL" that contain the string "404" or "303" but not both.

| set diff [search 404 | fields url] [search 303 | fields url]

Example 2:

Return all urls that have 404 errors and 303 errors.

| set intersect [search 404 | fields url] [search 303 | fields url]

Note: When you use the fields command in your subsearches, it does not filter out internal fields by default. If you do not want the set command to compare internal fields, such as the _raw or _time fields, you need to explicitly exclude them from the subsearches:

| set intersect [search 404 | fields url | fields - _*] [search 303 | fields url | fields - _*]

See also

append, appendcols, appendpipe, join, diff