How Splunk uses SOAP

How Splunk uses SOAP

Splunk's SOAP API is how the splunkweb process communicates with splunkd process. Anything you can do through Splunk's Web interface, you can do through SOAP. You can make calls directly by building complete SOAP messages via the front-end of your choice, or use the Python Control Layer interface.

Python Control Layer

The Python Control Layer (PCL) allows you to send SOAP requests from your Python code without having to generate the full call yourself. You can still use SOAP directly, but many operations are available in the PCL and more will be added in the future. All of the data input methods in the PCL take a (more or less) standardized Python dictionary object and, in most cases, return a dictionary as well.

As for the methods themselves, they follow the convention:

def <module>_<operation>(args, fromCLI)

where <module> is the input module (e.g. "tail" or "fifo") and <operation> is the operation that is to be performed with the module (e.g. "addFile" or "list"). args is a Python dictionary object. fromCLI is a boolean variable that is set to True if the method is called from the command line tool, but is otherwise defaulted to False for calls from other consumers, such as the search interface.

The PCL Data Input Methods can be found in the control_api.py module, located in $SPLUNK_HOME/lib/python2.4/site-packages/splunk/clilib. To use this module, import the following into your Python code:

control_api.py

cli_common.py

control_exceptions.py

control_api.py defines most of the interface, cli_common.py contains authorization functions and control_exceptions.py contains the exceptions used by the PCL. This directory also contains many additional source files that actually implement the interface. All the Python code that Splunk uses is in these files for you to examine. As more functionality is folded into the PCL, methods will be appropriately exposed through control_api.py.

You can find the list of supported commands in control_api.py (search for "cList" around line 720.) This file defines the interface exactly as Splunk uses it, so this is always the most accurate source for information. The list looks like this:

cList.addCmd(SplunkCmd("add",    "user",             user_add, "username" , proReq = True))
cList.addCmd(SplunkCmd("add",    "fifo",             fifo_add, "source"))

The argument "proReq = True" indicates that this function is only available for a Splunk Professional server. Any request to a Pro server must be authenticated by passing a valid authentication token with the request. Additional details can be found by looking at the function definitions elsewhere in the file or the implementations in the *.py files.

All methods take a dictionary (set of key-value pairs). Methods usually have one or more required key-value pairs and zero or more optional key-value pairs. All keys are required to be in lower-case. There is no case restriction on the values. If a module is unable to find a required key-value pair in the incoming dictionary, an exception will be raised. If the method finds a key-value pair that is not in its set of required or optional key-values, an exception will be raised.

On error the methods will raise exceptions, so calls to the PCL should be wrapped in try/except blocks for error handling purposes.

Most methods will return a dictionary on completion. In most cases, an empty dictionary on return signifies a successful call. In certain cases, the dictionary will contain a "serverRestartReq" key-value signifying that splunkd must be restarted for changes to take effect.

Environment

Splunk requires a number of environment variables to be set, such as SPLUNK_HOME, and the correct PATH to be able to locate splunk files. Source the file setSplunkEnv in the splunk bin directory (usually /opt/bin/splunk) to set up the necessary environment. As long as the environment is correct, you can run your Python code from anywhere on your filesystem.

source /opt/splunk/bin/setSplunkEnv

Authentication

Every call requires an authentication token be passed in the dictionary, even for the free server which does not support users or authentication. In this situation, use a dictionary with an empty string for the authstr.

All requests to a Splunk Professional server must be authenticated, otherwise they will fail. To do this, first call getAuthInfo with a valid username and password. The server responds with an authentication token to be used for subsequent requests. This token is good until you submit a logout request or the next time splunkd is restarted.

Example

This is a minimal example of using the PCL to generate a SOAP call. It has no authentication or error handling but shows the basic structure of a PCL call (in this case to a free server.) It generates a SOAP request for function tail_list and prints the returned dictionary containing a list of all the tails currently configured in the server. Note that it calls the version of Python supplied with Splunk and not any that may be installed on the system. Use Splunk's version of Python to avoid problems with incompatible versions.

#!/opt/splunk/bin/python -u
import splunk.clilib.control_api as ca
argsDict = {
  "authstr" : ""
}
results = ca.tail_list(argsDict)
print results

The output looks like this:

{'dirs': ['/var/log/httpd'], 'dynamic': ['/var/log/httpd/error_log', '/var/log/httpd/access_log'], 'static': ['/var/log/system.log']}

For this minimal configuration, you can see there is one directory (/var/log/httpd) that contains two files (error_log and access_log) and one additional file specified by name (system.log.)

See the following page for more complete examples of making PCL calls from your Python code.

• Was this topic useful?
• Post a comment