Splunk App for Stream REST API

Introduction

The Splunk App for Stream provides support for configuration and management of the Splunk Stream Forwarder. It also provides visualization of performance, error, and log data.

Streams

A stream represents protocol specific data, such as headers, payload, metrics, etc. For example, you can use HTTP stream to send HTTP protocol data to a Splunk index.

The following REST operations are supported for streams as a RESTful resource:

GET /streams

Returns a list of all configured streams.

Request Parameters

No parameters for this request.

Output

Returns a list of streams in json format.

Response Codes

Status Code	Description
200	OK

Response Headers

Content-Language:en-us
Content-Type:application/json
Server:CherryPy/3.1.2 WSGI Server

Example

http://localhost:8000/en-us/custom/splunk_app_stream/streams

[
    {
        "enabled": true,
        "extras": {
            "eventType": "clickstream.http-event"
        },
        "fields": [
            {
                "aggType": "sum",
                "desc": "total number of visitor sessions",
                "enabled": true,
                "name": "sessions",
                "term": "clickstream.new-session"
            },
            {
                "aggType": "sum",
                "desc": "total number of page views",
                "enabled": true,
                "name": "pages",
                "term": "clickstream.new-page"
            }, …

        ],
        "id": "http_metrics",
        "name": "http",
        "stream_type": "agg_event"
    },
    {
        "enabled": true,
        "extras": {
            "event": "test1",
            "eventType": "clickstream.http-event"
        },
        "fields": [
            {
                "aggType": "key",
                "desc": "server IP address",
                "enabled": true,
                "name": "server",
                "term": "clickstream.s-ip"
            },
            {
                "aggType": "key",
                "desc": "server port number",
                "enabled": true,
                "name": "port",
                "term": "clickstream.s-port"
            }, …
 
        ],
        "id": "tcp_metrics",
        "name": "TCP metrics aggregated by service (server/port)",
        "streamType": "agg_event",
        "stream_type": "agg_event"
    }
]

PUT /streams/{stream_id}

Updates a specific stream with {stream_id}:

Authentication

Requires csrf cookies and X-CSRFtoken in the header when using Splunk web portal to update.

Output

Returns updated stream details for {stream_id} in json format.

Response Headers

Content-Language:en-us
Content-Type:application/json
Server:CherryPy/3.1.2 WSGI Server

Example

http://localhost:8000/en-us/custom/splunk_app_stream/streams/http_metrics

Cookie:django_csrftoken_8000=wQHQXZ7eSDFbySaphIk1E90qsKvOHGhm; 
session_id_8000=8a96658d2e0b79d37417c8c41187ff6359e1a2fb; 
splunkweb_csrf_token_8000=5727260219993570948
X-CSRFtoken:wQHQXZ7eSDFbySaphIk1E90qsKvOHGhm

Request Payload

Complete resource i.e., all the fields should be sent in the request payload:
{
"id": "http_metrics",....
  "enabled": false,
  "fields":   [  {
      "name": "cached", 
      "aggType": "sum", 
      "term": "clickstream.cached", 
      "desc": "total number of HTTP transactions that had cached responses",
      "enabled": false
    }.....
  ]
}

Sample Response

{
    "enabled": false,
    "extras": {
        "eventType": "clickstream.http-event"
    },
    "fields": [
        {
            "aggType": "key",
            "desc": "IP address of HTTP server",
            "enabled": true,
            "name": "server",
            "term": "clickstream.s-ip"
        },
        {
            "aggType": "sum",
            "desc": "total number of visitor sessions",
            "enabled": true,
            "name": "sessions",
            "term": "clickstream.new-session"
        },
        {
            "aggType": "sum",
            "desc": "total number of page views",
            "enabled": true,
            "name": "pages",
            "term": "clickstream.new-page"
        },
        {
            "aggType": "sum",
            "desc": "total octets from client to server (ingress)",
            "enabled": true,
            "name": "cs_bytes",
            "term": "clickstream.cs-bytes"
        },
        {
            "aggType": "sum",
            "desc": "total octets from server to client (egress)",
            "enabled": true,
            "name": "sc_bytes",
            "term": "clickstream.sc-bytes"
        },
        {
            "aggType": "sum",
            "desc": "sum of measurements from start to end of each HTTP transaction (microsec)",
            "enabled": true,
            "name": "time_taken",
            "term": "clickstream.time-taken"
        },
        {
            "aggType": "sum",
            "desc": "sum of measurements from first to last request packet of each HTTP transaction (microsec)",
            "enabled": true,
            "name": "req_send_time",
            "term": "clickstream.cs-send-time"
        },
        {
            "aggType": "sum",
            "desc": "sum of time taken for the server to acknowldge receipt of HTTP requests (microsec)",
            "enabled": true,
            "name": "req_ack_time",
            "term": "clickstream.cs-ack-time"
        },
        {
            "aggType": "sum",
            "desc": "sum of measurements from last request to first response packet of each HTTP transaction (microsec)",
            "enabled": true,
            "name": "reply_time",
            "term": "clickstream.sc-reply-time"
        },
        {
            "aggType": "sum",
            "desc": "sum of measurements from first to last response packet of each HTTP transaction (microsec)",
            "enabled": true,
            "name": "rsp_send_time",
            "term": "clickstream.sc-send-time"
        },
        {
            "aggType": "sum",
            "desc": "sum of time taken for the client to acknowldge receipt of HTTP responses (microsec)",
            "enabled": true,
            "name": "rsp_ack_time",
            "term": "clickstream.sc-ack-time"
        },
        {
            "aggType": "sum",
            "desc": "sum of round trip time measurements between client & agent (microsec)",
            "enabled": true,
            "name": "client_rtt_sum",
            "term": "clickstream.cp-rtt-sum"
        },
        {
            "aggType": "sum",
            "desc": "count of round trip time measurements between client & agent",
            "enabled": true,
            "name": "client_rtt_count",
            "term": "clickstream.cp-rtt-packets"
        },
        {
            "aggType": "sum",
            "desc": "sum of round trip time measurements between server & agent (microsec)",
            "enabled": true,
            "name": "server_rtt_sum",
            "term": "clickstream.ps-rtt-sum"
        },
        {
            "aggType": "sum",
            "desc": "count of round trip time measurements between server & agent",
            "enabled": true,
            "name": "server_rtt_count",
            "term": "clickstream.ps-rtt-packets"
        },
        {
            "aggType": "sum",
            "desc": "total number of HTTP transactions that were refused by the server",
            "enabled": true,
            "name": "refused",
            "term": "clickstream.refused"
        },
        {
            "aggType": "sum",
            "desc": "total number of HTTP transactions that were canceled by the client",
            "enabled": true,
            "name": "canceled",
            "term": "clickstream.canceled"
        },
        {
            "aggType": "sum",
            "desc": "total number of HTTP transactions that had cached responses",
            "enabled": false,
            "name": "cached",
            "term": "clickstream.cached"
        }
    ],
    "id": "http_metrics",
    "name": "http",
    "stream_type": "agg_event"
}

To create a custom stream, use POST with no <id> in the URI. To delete a custom stream, use DELETE with <id> in the URI.

Capture IP Addresses whitelist/blacklist

Whitelist/blacklist of capture IP addresses to allow or disallow capturing traffic.

The following REST operations are supported for capture IP addresses as a RESTful resource:

GET /captureIpAddresses

Returns all capture IP addresses.

Request Parameters

No parameters for this request.

Output

Returns a list of both whitelisted and blacklisted IP addresses in json format.

Response Codes

Status Code	Description
200	OK

Response Headers

Content-Language:en-us
Content-Type:application/json
Server:CherryPy/3.1.2 WSGI Server

Example

http://localhost:8000/en-us/custom/splunk_app_stream/captureipaddresses

[
    {
        "id": "blacklist",
        "ipAddresses": [“^192\.168\.$”],
        "updatedBy": "admin"
    },
    {
        "id": "whitelist",
        "ipAddresses": [],
        "updatedBy": "admin"
    }
]

PUT /captureIpAddresses/{whitelist/blacklist}

Updates whitelist or blacklist of capture IP addresses.

Authentication

Requires csrf cookies and X-CSRFtoken in the header when using Splunk web portal to update.

Output

Returns the updated whitelist/blacklist of IP addresses in json format.

Response Codes

Status Code	Description
200	OK

Response Headers

Content-Language:en-us
Content-Type:application/json
Server:CherryPy/3.1.2 WSGI Server

Example

http://localhost:8000/en-us/custom/splunk_app_stream/captureipaddresses/whitelist

Cookie:django_csrftoken_8000=wQHQXZ7eSDFbySaphIk1E90qsKvOHGhm; 
session_id_8000=8a96658d2e0b79d37417c8c41187ff6359e1a2fb; 
splunkweb_csrf_token_8000=5727260219993570948
X-CSRFtoken:wQHQXZ7eSDFbySaphIk1E90qsKvOHGhm

Request Payload

{   
 "ipAddresses": [
        "^192\.168\.*\.$"
    ]
}

Sample Response

{
    "id": "whitelist",
    "ipAddresses": [
        "^192\\.168\\.*\\.$"
    ]
}