Search Manual

Write a search command

A custom search command is just a Python script that reads data in and writes data out. This topic discusses how your Python script should handle inputs and arguments.

The search command script should be should be:

• Located in the appropriate $SPLUNK_HOME/etc/apps/<app_name>/bin/ directory. Most of the scripts that ship with Splunk are associated with the Search app and are stored in $SPLUNK_HOME/etc/apps/search/bin. (You can refer to scripts in this directory for examples.)
• Named <command_name>.py. That is, the script filename should be the command name that you will invoke in your Splunk search. Search command names can consist only of alphanumeric (a-z, A-Z, and 0-9) characters. New commands should not have the same name of any existing commands. Read more about the naming conventions for custom commands in the "Search command style guide".

Types of commands

A search command can be a generating command, a streaming command, or one that generates events.

• A generating command is a command that retrieves (or generates) events from an index(es). These commands are used at the beginning of the search pipeline. They don't expect/require an input (whether it's events or search results) and instead are invoked with a leading pipeline. Examples of generating commands include search (when used at the beginning of the pipeline), inputcsv, inputlookup, and metadata.
• A streaming command is one that applies a transformation to each event and writes out the results in a chunked manner instead of all at once. Examples of streaming commands include eval (which is used to add one or more fields to each event), where, and streamstats. A non-streaming command expects to have all the data available as an input before it operates or reduces that data into the output. Examples of non-streaming commands include stats, top, and timechart.
• A command that generates events is one that returns data in a streaming manner. Examples of these commands include search, eval, and where.

The default initial search operation generates events and the retrieved events will display in the events viewer tab in Splunk Web. Some generating commands (such as inputcsv, inputlookup, or metadata) don't generate events, and therefore the events viewer will not show anything. Certain operators (such as eval, sort, dedup, cluster, and where) preserves or retains events while other commands (such as stats, top, and timechart) do not.

You can describe what type of command your custom search command is with the parameters streaming and generating in commands.conf. You can also specify whether it retains or transforms events with the retainevents parameter. Read "Add the custom command to Splunk" for more information. Find all the configurable settings in the commands.conf reference in the Admin Manual.

Handling inputs

The input to the script should be formatted in pure CSV or in Intersplunk, which is a header section followed by a blank line followed by pure CSV body.

The simplest way to interpret your script input is to use splunk.Intersplunk.readResults, which takes 3 optional parameters and returns a list of dicts (which represents the list of input events). The optional parameters are 'input_buf', 'settings', and 'has_header':

• 'inputbuf' is where to read input from, and if it is None (by default), it is assumed to be sys.stdin.
• 'settings' is expected to be a dict where we will store any information found in the input header (default = None, means don't record the settings).
• 'has_header' indicates whether or not we expect an input header and is True by default.

To indicate whether or not your script expects a header, use the 'enableheader' key. The 'enableheader' key defaults to true, which means that the input will contain the header section and you are using the Intersplunk format.

If your script does not expect a header section in the input (enableheader is false), you can directly use the Python csv module to read the input. For example:

import csv

r = csv.reader(sys.stdin)
for l in r:
...

The advantage of this usage is that you can break at any time in the for loop, and only lines in the input that you had iterated to at that point will have been read into memory. This leads to much better performance for some usage cases.

Sending output

You can also use Intersplunk to construct your script's output. splunk.Intersplunk.generateErrorResults takes a string and writes the correct error output to sys.stdout. splunk.Intersplunk.outputResults takes a list of dict objects and writes the appropriate CSV output to sys.stdout.

To output data, add:

splunk.Intersplunk.outputResults(results)

The output of your script is expected to be pure CSV. For an error condition, simply return a CSV with a single "ERROR" column and a single row (besides the header row) with the contents of the message.

Handling errors

The arguments that are passed to your script (in sys.argv) will be the same arguments that are used to invoke your command in the search language unless your script has supports_getinfo = true. The supports_getinfo key indicates that the first argument to your script will either be __GETINFO__ or __EXECUTE__. This allows you to call the script with the command arguments at parse time to check for syntax errors before any execution of the search. Errors at this time will short circuit any real execution of the search query. If called with __GETINFO__, this also allows you to dynamically specify the properties of your script (such as streaming or not) depending on your arguments.

If your script has supports_getinfo set to 'true', you should first make a call like:

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)

This call will strip the first argument from sys.argv and check if you are in GETINFO mode or EXECUTE mode. If you are in GETINFO mode, your script should use splunk.Intersplunk.outputInfo() to return the properties of your script or splunk.Intersplunk.parseError() if the arguments are invalid.

The definition of outputInfo() and its arguments is as follows:

def outputInfo(streaming, generating, retevs, reqsop, preop, timeorder=False)

You can also set these attributes in commands.conf.