About the search processing language
When you hear about the Splunk search processing language (SPL), you may have heard the terms distributable, streaming, generating, and transforming used to describe the types of search commands. This topic describes what these terms mean and lists the commands that fall into each category.
Search processing language components
The search processing language encompasses all the search commands and their functions, arguments and clauses. Search commands tell Splunk Enterprise what to do to the events you retrieved from the indexes. For example, you need to use a command to filter unwanted information, extract more information, evaluate new fields, calculate statistics, reorder your results, or create a chart.
Some search commands have functions and arguments associated with them. Use these functions and their arguments to specify how the commands act on your results and/or which fields they act upon. For example, use functions to format the data in a chart, describe what kind of statistics to calculate, and specify what fields to evaluate. Some commands also use clauses to specify how to group your search results.
Types of search commands
There are four broad categorizations for all the search commands: distributable streaming, stateful streaming, transforming, generating.
A streaming command operates on each event returned by a search. A distributable streaming command runs on the indexer and can be applied to subsets of indexed data in a parallel manner. For example, the regex command is streaming; it extracts fields and adds them to events at search time.
Distributable streaming commands include: bucket (if it's called with an explicit
span), convert, eval, extract (kv), fields, lookup (if not local=t), mvexpand, multikv, rename, regex, replace, rex, search, strcat, tags, typer, and where.
A centralized streaming command applies a transformation to each event returned by a search, but unlike distributable streaming commands, it only works on the search head. You might also hear the term "stateful streaming" to describe these commands.
A transforming command orders the results into a data table, that is, it "transforms" the specified cell values for each event into numerical values that Splunk can use for statistical purposes. Transforming commands are not streaming. Also, they are required to transform search result data into the data structures required for visualizations such as column, bar, line, area, and pie charts.
A generating command is one that fetches information without any transformations. Generating commands are either event-generating (distributable or centralized) or report-generating and, depending on which they are, will return an events list or a table of results. Generating commands are usually invoked at the beginning of the search and with a leading pipe. That is, there cannot be a search piped into a generating command. The exception to this is the search command, because it is implicit at the start of a search and does not need to be invoked.
There are a handful of commands that do not fit into these categories. These commands are non-reporting, not distributable, and not streaming: sort, eventstats, some modes of dedup, and some modes of cluster.
The search processing language syntax
This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14