Export data using the Splunk SDKs

Splunk Software Development Kits (SDKs) enable software developers to create Splunk apps using common programming languages. Splunk SDKs let you integrate Splunk deployments with third-party reporting tools and portals, include search results in your application, and extract high volumes of data for archival purposes. To use Splunk SDKs, you should be proficient in SDK knowledge and development.

Splunk offers SDKs for Python, Java, and JavaScript. When you run an export-search in these SDKs, the search runs immediately, it does not create a job for the search, and it start streaming results immediately.

The Splunk SDKs are built on top of the Splunk REST API. They provide a simpler interface for the REST API endpoints. With fewer lines of code, you can write applications that can:

• Create and run authenticated searches
• Add data
• Index data
• Manage search jobs
• Configure Splunk

For more information about the Splunk SDKs, read "Overview of the Splunk SDKs" in the Splunk Developer Portal.

Use Python SDK to export data

The Splunk SDK for Python lets you write Python applications that can interact with Splunk deployments. Export searches using the Python SDK can be run in historical mode and real-time mode. They start right away, and stream results instantly, letting you integrate them into your Python application.

Perform an export search using the Python SDK. This script has been made cross-compatible with Python 2 and Python 3 using python-future.

1. Set the parameters of what you wish to search. The following example sets the parameters as an export search of splunklib in the last hour.

sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "lib"))
import splunklib.client as client
import splunklib.results as results
from __future__ import print_function

Note that sys.path.insert adds lib to the path so that the app calls the version of splunklib installed with this app, which you should store in the /lib directory of the app, as detailed in The directory structure of a Splunk App in Splunk developer docs.

2. Change or acquire these values, as necessary.

HOST = "localhost"
PORT = 8089
USERNAME = "admin"
PASSWORD = "changeme"

3. Run a normal-mode search.

service = client.connect(
    host=HOST,
    port=PORT, 
    username=USERNAME,
    password=PASSWORD)

rr = results.ResultsReader(service.jobs.export("search index=_internal earliest=-1h | head 5"))

4. Get the results and display them using the ResultsReader.

for result in rr:
    if isinstance(result, results.Message):
        # Diagnostic messages might be returned in the results
        print('%s: %s' % (result.type, result.message))
    elif isinstance(result, dict):
        # Normal events are returned as dicts
        print(result)
assert rr.is_preview == False

Use Java SDK to export data

The Java SDK is able to conduct and export searches while using Java.

To perform an export search using the Java SDK, run the following example in the /splunk-sdk-java directory using the CLI:

java -jar dist/examples/export.jar main --username="admin" --password="changeme"

The Export application exports the "main" index to export.out, which is saved to the current working directory. If you want to run this application again, delete export.out before you try again. If you do not do this, you will get an error.

Here is a different CLI example of the Java SDK. It shows how to include a search query and change the output format to JSON.

java -jar dist/examples/export.jar main --search="search sourcetype=access_*" json

Use JavaScript Export to export data

The Javascript Export endpoint can export Splunk data in the Javascript framework. Though the Splunk Javascript SDK does not currently support the Javascript Export endpoint, you can use a node javascript (.js) application request to export data.

To perform an export search using the Javascript Export endpoint:

1. Load the request module. Request is designed to be the simplest way to make an http/https call.

var request = require('request');

2. Call get to issue a GET request. Enter the following parameters:

• strictSSL – When set to false, strictSSL tells the request to not validate the server certificate returned by your Splunk deployment, which by default is not a valid certificate.
• uri – Provide the uri of the Splunk host along with the path for the export endpoint. A JSON response is specified in the query string.
• qs – Set qs to supply the search parameter. By passing it this way, you do not have to URI encode the search string.

request.get(
    {
        strictSSL: false,
        uri: 'https://localhost:8089/servicesNS/admin/search/search/jobs/
              export?output_mode=json',
        qs: {
            search: 'search index=_internal'
        }
    }
)

3. Call auth to use HTTP Basic Auth and pass your Splunk username and password.

.auth('admin', 'changeme', false)

4. Pipe the results to stdout.

.pipe(process.stdout);