SDKs

SDKs

Splunk currently has SDKs available on Splunk Lab's googlecode page.

For a full list of the Splunk-implemented methods/classes for each SDK, download this excel spreadsheet. Please note that not all endpoints have wrappers for each SDK. The currently available wrappers include:

• Search
• Authentication
• Properties (or configuration file access)

Python SDK

Splunk ships with an embedded Python SDK. The internal SDK is also used by the web application framework inside of the splunkd process.

1. Source Splunk to load the correct Python:

source $SPLUNK_HOME/bin/setSplunkEnv

Note: $SPLUNK_HOME is the location of your Splunk install. For example, opt/splunk.

2. You can load available Python modules via the following command in your $SPLUNK_HOME/bin directory (or the splunk/ directory, if you're using the Python-External SDK):

pydoc -p 8080

This loads all the available modules into http://localhost:8080. SDK modules are located at http://localhost:8080/splunk.html.

3. Or, to see all possible python commands in the full Splunk server (from the $SPLUNK_HOME/bin/ dir):

splunk cmd python

4. Type help() to get to the interactive Python help.

5. Type modules for a list of the available modules, or help(<module>) for help on a specific module.

Examples

Start up Python and try getting an auth key:

# source /opt/splunk/bin/setSplunkEnv 
# python 
Python 2.5.1 (r251:54863, Apr  4 2008, 00:16:06) 
[GCC 4.0.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from splunk import auth, search
>>> import time
>>> 
>>> auth.getSessionKey('admin', 'changeme')
'43d7ea46ff602238ca5d1de56e17f692'
>>> 

Here's an example that gets a session key, then performs a search for events from the last minute. The search is performed synchronously, so your code will block until Splunk is done returning results. Stick this code in something like _example.py_:

from splunk import auth, search
import time
auth.getSessionKey('admin','changeme')
job = search.dispatch('search * startminutesago=1')
# this will stream events back until the last event is reached
for event in job:
	print event
	
job.cancel()

Running this outputs the raw events from the last minute:

# source /opt/splunk/bin/setSplunkEnv 
# python example.py 
111.111.111.111 - - [17/Jun/2008:13:26:09 -0500] "GET http://photos.zoto.com/kordless/img/28/40aab3c632b6fc2215cc850545793c31.jpg HTTP/1.0" 200 19429 "http://splunk.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM; .NET CLR 2.0.50727; InfoPath.1)"
.
.
.

Limiting Output to Extracted Fields

This next example limits output to a particular field which was extracted at index time. In this example Splunk is extracting the 'clientip' field:

from splunk import auth, search
import time
auth.getSessionKey('admin','changeme')
job = search.dispatch('search * startminutesago=1')
# this will stream events back until the last event is reached
for event in job:
	print event['clientip']
job.cancel()

Running this code outputs only the IP addresses that were extracted:

root@ulysses [~]# python example.py 
111.111.111.111
222.222.222.222
.
.
.

Limiting Output to Fields Extracted at Search Time

If Splunk hansn't extracted a particular field, you can use the _rex_ command to extract them at search time:

* startminutesago=1 | rex field=_raw "(?<imageid>[0-9a-f]{32})"

This search string assumes an MD5 exists in the event stream. Use your own regular expressions to extract a custom field from your own data.

You can test your rex extractions with the Splunk UI to ensure you are getting back the correct results before starting to code. To see the extracted field in the Splunk UI, you'll need to select extracted fields from the fields pulldown:

[1]

Be aware that the _events_ object being used above returns un-transformed data. In this example, the _rex_ command is a transforming command and requires using the _results_ object type instead of _events_.

You'll need to wait on Splunk to finish the search before you get back these transformed results. Splunk provides a method for checking to see if a job is done or not, and we use it to hang out until the results are back and transformed:

from splunk import auth, search
import time
auth.getSessionKey('admin','changeme')
job = search.dispatch('search * startminutesago=1 | rex field=_raw "(?<imageid>[0-9a-f]{32})" | where imageid > ""')
# at this point, Splunk is running the search in the background; how long it
# takes depends on how much data is indexed, and the scope of the search
# wait until the job has completed before trying to access job
while not job.isDone: 
    time.sleep(1)
# this will iterate through the completed results - with transforms applied
for result in job.results:
	print result['imageid']
job.cancel()

Notice we use a _where_ clause to filter out results that don't contain an extracted _imageid_ field. We do this because some events may not provide a match to our regular expression!

# python example.py 
2387d1e5d205d5d9e803e6535f66aacc
71d6cad91460f5f9873fb57c5ebcf446
2e63a453e5292da64292deb724a7bb9b
d518ed5fb7e21548b5efbe8f7d2c232b
ae0b6c614a69b579aae4a01ffc4a07ba
f8efa202eb3dbc8d50f298ee762d683c
f3422c85de5e843c0c43c91ebe89aac3
.
.
.

• Was this topic useful?
• Post a comment