Write a Python search command
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Write a Python search command
A custom search command is just a Python script that reads data in and writes data out. There are two subtypes of custom search commands: streaming or non-streaming. A streaming command applies a transformation to each event and writes out the result, such as adding a field to each event. A non-streaming command expects to have all the data before it operates or reduces the data into the output.
This topic discusses how your Python script should handle inputs and arguments.
The search command script should be:
- Located in
Search command names can consist only of alphanumeric (a-z, A-Z, and 0-9) characters. New commands should not have the same name of any existing commands.
The input to the script should be in pure CSV format or in Intersplunk format, which is a header section followed by a blank line followed by pure CSV body.
To indicate whether or not your script expects a header, use the 'enableheader' key. The 'enableheader' key defaults to true, which means that the input will contain the header section and you are using the Intersplunk format.
If 'enableheader' is false, your script does not expect a header section and the input will be pure CSV. In this case, you can directly use the Python CSV module to read the input. For example:
r = csv.reader(sys.stdin)
for l in r:
The advantage of this usage is that you can break at any time in the for loop, and only lines in the input that you had iterated to at that point will have been read into memory. This leads to much better performance for some usage cases.
The output of your script is expected to be pure CSV. For an error condition, simply return a CSV with a single "ERROR" column and a single row (besides the header row) with the contents of the message.
Another method of interpreting your script input is to use
splunk.Intersplunk.readResults, which takes 3 optional parameters and returns a list of dicts (which represents the list of input events). The optional parameters are 'input_buf', 'settings', and 'has_header':
- 'inputbuf' is where to read input from, and if it is None (by default), it is assumed to be
- 'settings' is expected to be a dict where we will store any information found in the input header (default = None, means don't record the settings).
- 'has_header' indicates whether or not we expect an input header and is True by default.
You can also use Intersplunk to construct your script's output.
splunk.Intersplunk.generateErrorResults takes a string and writes the correct error output to
splunk.Intersplunk.outputResults takes a list of dict objects and writes the appropriate CSV output to
The arguments that are passed to your script (in sys.argv) will be the same arguments that are used to invoke your command in the search language.
The exception is if your script has
supports_getinfo = true. The
supports_getinfo key indicates that the first argument to your script will either be
__GETINFO__ or __EXECUTE__. This allows you to call the script with the command arguments at parse time to check for syntax errors before any execution of the search. Errors at this time will short circuit any real execution of the search query. If called with
__GETINFO__, this also allows you to dynamically specify the properties of your script (such as streaming or not) depending on your arguments.
If your script has
supports_getinfo set to 'true', you should first make a call like:
(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
This call will strip the first argument from sys.argv and check if you are in GETINFO mode or EXECUTE mode. If you are in GETINFO mode, your script should use
splunk.Intersplunk.outputInfo() to return the properties of your script or
splunk.Intersplunk.parseError() if the arguments are invalid.
Following is the definition of
outputInfo() and its arguments. You can also specify each of these arguments statically within commands.conf.
def outputInfo(streaming, generating, retevs, reqsop, preop, timeorder=False)
Is your command streamable?
streaming = true indicates that your script can be applied separately for each chunk of results processed in the search pipeline. Otherwise, your script will only be called a single time with all of the input results that it will ever see.
If your script is not streaming, you may specify a search (that contains only streaming commands) to be executed before your script, if your script is the very first non-streaming command in the pipeline or if you have
requires_preop set to 'true' (it's 'false' by default).
Does you command generate new events? A generating command is one that must be the first command specified in a search. Generating commands do not expect any input and generating output that depends only on command line arguments.
Does your command retain events (sort, dedup, cluster) or does it transform (stats) them? This argument is
This argument indicates whether this script, if given 'events' as input will return 'events' as output. By default this is 'false', meaning that the timeline will never represent the output of this command. In general, if you retain the '_raw' and '_time' fields, you can set
retevs to 'true'.
Does your command require pre-streaming operations? This argument is
Basically, this argument indicates whether the string in the 'preop' variable must be executed, regardless if this script is the first non-streaming command in a search pipeline or not.
This argument is the
streaming_preop key in
reqsop = true, this argument is the string that denotes the requested pre-streaming search string.
generating = true, does your command generate events in descending time order, or does your command change the time order of events?
This argument represents both
overrides_timeorder indicates whether or not, if the input to this script is in descending time order, the output will also be in descending time order.
generates_timeorder applies when this script is a generating command and indicates if this script will issue output in descending time order (and the output is always regardless of input).
There are two examples in this chapter to help you understand how to write your own custom search command:
- The shape command categorizes your events with descriptive words based on line length and character count.
- The iplocation command is a Splunk command that determines location information from the IP addresses in your raw event data.