Splunk® Enterprise

REST API Reference Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

KV store endpoint descriptions

This introduction describes syntax common to all App KV Store REST endpoints. For a description of how to use the App KV Store REST API, see App Key Value Store.

Note: Username and password authentication is required for most endpoints and REST operations. Additional capability or role-based authorization may also be required, particularly for POST or DELETE operations.

App KV store REST API features

App KV store provides:

  • Create, read, list and delete EAI operations on a collection.
  • Indices create, read, list and delete indices, for a given collection. All collection methods are EAI-compatible. These are not regular EAI operations because the indices are stored as attributes on the EAI entity. If a user does not have a permission for a given collection, these operations are prohibited.
  • The ability to insert, read, update and delete a single key and key value, for a given collection, provided you have permission to modify the collection.
    • Duplicate keys are not allowed.
    • All updates are wholesale updates.
    • Partial value update is prohibited.
  • The ability to query key-value pairs, for a given collection. For example, to find all the documents that have a host field with a value, use the query {"host": "bar.com"}. Only the following operations are permitted: $gt, $gte, $lt, $lte, $ne, $and, $or, and $not. The API also supports mechanisms for projection, sorting and pagination.
  • The ability to do batch reads/inserts/updates, for a given collection. This supports high-volume operations and limits network traffic. These operations have the same semantics as their individual counterparts.

Limits

  • 16MB per record
  • 1KB per indexed field

Collections

A Collection is a container of "documents" or "values". Most documents have a similar structure, although structure is not enforced. For example, a Collection might have Notable Events or entities.

Collections are declared in a collections.conf configuration file, where a stanza is a single collection.

Collections are Splunk platform entities, which means they are permissioned using the Splunk platform RBAC/ACL system. One reason of exposing Collections as entities is to ensure they are properly managed. But, the implementation permits only app-level collections. This means that collections can be defined only in the etc/apps/<app>/<default|local> directory, not in etc/users/.... Attempts to create a user-specific Collection using the API fail and Collections created manually in the .conf files are ignored.

Data

A collection stores key-value pairs consisting of a user-specified or autogenerated key and a JSON document-formatted value. A JSON document/object of values is enforced for simplicity instead of using integer or string raw values. For example, {"_key": "10.0.0.1", host: "bar.com"} defines the key ("10.0.0.1") and the full JSON document value.

A key is fetched, updated, and deleted. Partial updates, including array append, are prohibited. You must set the whole document. Keys are unique within a collection and support only the basic JSON data types: boolean, string, number, object, array, and null.

A value is specified in the _key attribute of the document and, if one is not specified, a random value with a desirable sort order is automatically generated.

A collection defined at:

/servicesNS/nobody/search/storage/mycollection

can have operations performed on it using the following URLs:

  • /servicesNS/nobody/search/storage/mycollection
  • /servicesNS/itay/search/storage/mycollection
  • /servicesNS/mark/search/storage/mycollection

Namespaces are separate, and data are not interleaved. Wildcard operations on the <user> or <app> parts of the namespace are prohibited.

User-specific collections are not supported so all users, including nobody, share a single namespace and _key is unique across all namespaces.

Types

Defining a set of types for some fields in a collection is provided. If a field has no type defined, it uses the type submitted over the API. If a type conversion fails, the insert/update operation is aborted. For example, if the collections.conf file has the following collection 

[mycollection]
field.enabled = bool
field.data.range = cidr

the data inserted or updated in the collection have the fields enabled and range-converted to the applicable type: 

* array
* number
* boolean
* time
* string
* cidr

Items need to specified using the field prefix only if type needs to be enforced for the field. The array type and cidr types are not enforceable, so they do not need to be specified.

Arrays

Arrays do not change a nested key name. For example, the key "a.b.c" can refer to {"a": {"b": {"c": 1}}} or {"a": {"b": [{"c": 1}]}}.

Numbers

All strings/numbers/booleans are converted to doubles. If a number fails to convert, it inserts the string instead.

Booleans

All numbers/strings/booleans are converted to booleans. If a boolean fails to convert, it inserts the string instead. Epoch time can be a string or as a number. If it fails to convert, the string version is inserted.

Strings

Everything is converted to a string.

CIDR

All provided strings are converted to canonical CIDR strings. For example: 

127.0.0.1       -> 127.000.000.001/32
127.0.0.0/24    -> 127.000.000.000/24
127.0.0/24      -> 127.000.000.000/24

with similar canonicalization for IPv6: 

2001:db8::/96               -> 2001:0db8:0000:0000:0000:0000:0000:0000/096
2001:db8:0:0:0:0:0:ffff     -> 2001:0db8:0000:0000:0000:0000:0000:ffff/128

Queries

Combining the Data and Index patterns gives the ability to do basic querying. Queries are limited to the basic operators available in most databases, and at the moment do not expose in-array and not-in-array containers. A query may have the following operators:

  • gt
  • gte
  • lt
  • lte
  • eq
  • neq
  • or
  • and
  • not

Queries are permitted whether or not an index covers the query. If it the index does not cover the query, the query takes longer. For example, a query to find all Notable Events authored by userA or userB might look like: 

{ $or: [ { author: "userA" }, { author: "userB" } ] }

REST methods

The REST API for App KV Store implements the GET, POST, and DELETE methods described in this reference.

Defining KV Store lookups

For information on using the REST API to define or update a KV Store lookup, see data/transforms/lookups/ and data/transforms/lookups/{name}.

storage/collections/config

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config

Authentication: Required

Description

Create or list collections.

Method summary

Method Description Formats
GET List all collections. XML, JSON
POST Create a new collection. XML, JSON

GET storage/collections/config method detail

Example

Request parameters

None

Response data keys
Name Description
disabled Collection state:
true = disabled.
false = [Default] enabled.
profilingEnabled Profiling status of slow-running operations:
true = Profiling enabled.
false = (Default) Profiling disabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled = true.

POST storage/collections/config method detail

Example

Request parameters
Name Type Default Description
name
required 		String Collection name.
profilingEnabled Boolean A collections.conf file property. Affects profilingThresholdMs. Enable profiling of slow-running operations:
true = Enable profiling.
false = (Default) Disable profiling.
profilingThresholdMs Number Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled = true. This can have a performance impact on the collection.
0 = Log all slow-running operations.
Default = 100.


Response data keys
Name Description
disabled Collection state:
true = disabled.
false = [Default] enabled.
profilingEnabled Profiling status of slow-running operations, affecting profilingThresholdMs:
true = Profiling enabled.
false = (Default) Profiling disabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled = true.
Application usage

[ Top ]

storage/collections/config/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config/{collection}

Authentication: Required

Description

Manage a specific collection.

Method summary

Method Description Formats
DELETE Drop the {collection} collection. XML, JSON
GET Get {collection} collection information. XML, JSON
POST Update the {collection} collection. XML, JSON

DELETE storage/collections/config/{collection} method detail

Example

Request parameters

None

Response data keys

None

GET storage/collections/config/{collection} method detail

Example

Request parameters

None

Response data keys
Name Description
disabled Collection state:
true = disabled.
false = [Default] enabled.
field.<fieldType> Field type for field <fieldType>. One of:
array (implicit)
number
bool
string
cidr (implicit)
time
index.<indexName> Index definition named <indexName>.
profilingEnabled Profiling status of slow-running operations, affecting profilingThresholdMs:
true = Profiling enabled.
false = (Default) Profiling disabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled = true.

POST storage/collections/config/{collection} method detail

Example

Request parameters
Name Type Default Description
field.<fieldType> String Field type for field <fieldType>. One of:
array (implicit)
number
bool
string
cidr (implicit)
time
index.<indexName> String Index definition named <indexName>.
Response data keys
Name Description
disabled Collection state:
true = disabled.
false = [Default] enabled.
field.<fieldType> Field type for field <fieldType>. One of:
array (implicit)
number
bool
string
cidr (implicit)
time
index.<indexName> Index definition named <indexName>.
profilingEnabled Profiling status of slow-running operations, affecting profilingThresholdMs:
true = Profiling enabled.
false = (Default) Profiling disabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled = true.
Application usage

[ Top ]

storage/collections/data/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}

Authentication: Required

Description

Manage items of a collection.

Method summary

Method Description Formats
DELETE Delete item(s) in the {collection} collection. XML, JSON
GET Get items in the {collection} collection. XML, JSON
POST Insert an item in the {collection} collection. JSON

DELETE storage/collections/data/{collection} method detail

Example

Request parameters
Name Type Default Description
query JSON object Query JSON object.
Conditional operators: $gt, $gte, $lt, $lte, and $ne
Logical operators: $and, $or, and ,$not (invert conditional operators)
Examples:
query={"title":"Item"} (Select all documents with property title that has value Item)
query={"price":{"$gt":5}} (Select all documents with price greater than 5)
Response data keys

None

Application usage

The request deletes entries matching the query. If no query is provided, all entries in the collection are deleted.

GET storage/collections/data/{collection} method detail

Example

Request parameters
Name Type Default Description
fields String Comma-separated list of fields to include (1) or exclude (0). A fields value cannot contain both include and exclude specifications except for exclusion of the _id field. Examples:
fields=firstname,surname (Include only firstname, surname, and _id fields)
fields=firstname,surname,_id:0 (Include only the firstname and surname fields.
fields=address:0 (Include all fields except the address field)
limit Number Maximum number of items to return. Example:
limit=5 (Return five items)
skip Number Number of items to skip from the start. Example:
skip=10 (Skip first 10 items)
sort String Sort order. Examples:
sort=surname (Sort by surname, ascending)
sort=surname,firstname (Sort by surname, ascending, after firstname, ascending)
sort=surname:-1,firstname:1 (Sort by surname, descending, after firstname, ascending
sort=surname:1,first name (Sort by surname, ascending, after firstname, ascending
query JSON object Query JSON object.
Conditional operators: $gt, $gte, $lt, $lte, and $ne
Logical operators: $and, $or, and ,$not (invert conditional operators)
Examples:
query={"title":"Item"} (Select all documents with property title that has value Item)
query={"price":{"$gt":5}} (Select all documents with price greater than 5)
Response data keys

JSON document. Example: 

[ { <span style="color: #008000; font-weight: bold">"myKey"</span> : <span style="color: #BA2121">"abc"</span>, <span style="color: #008000; font-weight: bold">"_user"</span> : <span style="color: #BA2121">"nobody"</span>, <span style="color: #008000; font-weight: bold">"_key"</span> : <span style="color: #BA2121">"5410be5441ba15298e4624d1"</span> } ]
Application usage

Parameter order does not matter. Parameters are always applied in the order of sort, skip, and limit.

POST storage/collections/data/{collection} method detail

Example

Request parameters

JSON-formatted document.

Example: 

{ "name": "A" }
Response data keys

JSON-formatted document key.

Example: 

{
    "_key": "530bc62fc9a6577fdf13651f"
}
Application usage

The Content-Type header must be application/json.

The _id is autogenerated, if not manually specified.

[ Top ]

storage/collections/data/{collection}/{key}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}/{key}

Authentication: Required

Description

Manage a specific item of a collection.

Method summary

Method Description Formats
DELETE Delete item {key} in {collection}. XML, JSON
GET Get item in {collection} by {key} XML, JSON
POST Update the {key} item in {collection}. JSON

DELETE storage/collections/data/{collection}/{key} method detail

Example

Request parameters

None

Response data keys

None

GET storage/collections/data/{collection}/{key} method detail

Example

Request parameters

None

Response data keys

JSON-formatted {key} document.

Example: 

{"myKey" : "abc", "myOtherKey" : "uvwxyz", "_user" : "nobody", "_key" : "5410c8dc41ba15298e4624d5"}

POST storage/collections/data/{collection}/{key} method detail

Example

Request parameters

JSON-formatted document for {key}.

Example: 

{ "myKey": "fizz"}
Response data keys

JSON-formatted {key}.

Example: 

{"_key":"5410c8dc41ba15298e4624d5"}
Application usage

The header Content-Type must be application/json.

[ Top ]

storage/collections/data/{collection}/batch_save

https://<host>:<mPort>/servicesNS/storage/collections/data/{collection}/batch_save

Authentication: Required

Description

Perform multiple save operations.

Method summary

Method Description Formats
POST Perform multiple save operations. JSON

POST storage/collections/data/{collection}/batch_save method detail

Example

Request parameters

JSON array of documents to save.

Response data keys

JSON-formatted list of keys added.

Example: 

[ "5410c43241ba15298e4624d3", "5418c54e41ba152267763c01" ]
Application usage

Header Content-Type must be application/json.

If a document does not have an _key field or store does not have a matching document ID, an _key new document is created.

If an operation fails, the server stops batch processing and returns an HTTP status code other than 200.

[ Top ]

PREVIOUS
Knowledge endpoint examples 		  NEXT
KV store endpoint examples

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters