set
Description
Performs set operations on subsearches.
Syntax
The required syntax is in bold.
| set (union | diff | intersect) subsearch1 subsearch2
Required arguments
- union | diff | intersect
- Syntax: union | diff | intersect
- Description: Performs two subsearches, then executes the specified set operation on the two sets of search results.
Operation Description union
Returns a set that combines the results generated by the two subsearches. Provides results that are common to both subsets only once. diff
Returns a set that combines the results generated by the two subsearches and excludes the events common to both. Does not indicate which subsearch the results originated from. intersect
Returns a set that contains results common to both subsearches.
- subsearch
- Syntax: "[" <string> "]"
- Description: Specifies a subsearch. Subsearches must be enclosed in square brackets. For more information about subsearch syntax, see "About subsearches" in the Search Manual.
Usage
The set
command is an event-generating command. See Command types.
Generating commands use a leading pipe character and should be the first command in a search.
Results
The set
command considers results to be the same if all of fields that the results contain match. Some internal fields generated by the search, such as _serial, vary from search to search. You need to filter out some of the fields if you are using the set
command with raw events, as opposed to transformed results such as those from a stats
command. Typically in these cases, all fields are the same from search to search.
Output limitations
There is a limit on the quantity of results that come out of the invoked subsearches that the set
command receives to operate on. If this limit is exceeded, the input result set to the diff
command is silently truncated.
If you have Splunk Enterprise, you can adjust this limit by editing the limits.conf
file and changing the maxout
value in the [subsearch]
stanza. If this value is altered, the default quantity of results coming from a variety of subsearch scenarios are altered. Note that very large values might cause extensive stalls during the 'parsing' phase of a search, which is when subsearches run. The default value for this limit is 10000.
Only users with file system access, such as system administrators, can edit the configuration files. Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make the changes in the local directory.
See How to edit a configuration file.
If you have Splunk Cloud Platform and want to edit a configuration file, file a Support ticket.
Result rows limitations
By default the set
command attempts to traverse a maximum of 50000 items from each subsearch. If the number of input results from either search exceeds this limit, the set
command silently ignores the remaining events. By default, the maxout
setting for subsearches in limits.conf
prevents the number of results from exceeding this limit.
This maximum is controlled by the maxresultrows
setting in the [set]
stanza in the limits.conf file. Increasing this limit can result in more memory usage.
Only users with file system access, such as system administrators, can edit the configuration files. Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make the changes in the local directory.
See How to edit a configuration file.
If you have Splunk Cloud Platform and want to edit a configuration file, file a Support ticket.
Examples
Example 1:
Return values of "URL" that contain the string "404" or "303" but not both.
| set diff [search 404 | fields url] [search 303 | fields url]
Example 2:
Return all urls that have 404 errors and 303 errors.
| set intersect [search 404 | fields url] [search 303 | fields url]
Note: When you use the fields
command in your subsearches, it does not filter out internal fields by default. If you do not want the set
command to compare internal fields, such as the _raw
or _time
fields, you need to explicitly exclude them from the subsearches:
| set intersect [search 404 | fields url | fields - _*] [search 303 | fields url | fields - _*]
See also
sendemail | setfields |
This documentation applies to the following versions of Splunk Cloud Platform™: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403 (latest FedRAMP release), 9.2.2406
Feedback submitted, thanks!