KV store endpoint descriptions

This introduction describes syntax common to all app KV Store REST endpoints. For more information about the KV Store, see App Key Value Store on the Splunk developer portal.

REST API usage details

Review ACL information for an endpoint

To check Access Control List (ACL) properties for an endpoint, append /acl to the path. For more information see Access Control List in the REST API User Manual.

Authentication and Authorization

Username and password authentication is required for access to endpoints and REST operations.

Splunk users must have role and/or capability-based authorization to use REST endpoints. Users with an administrative role, such as admin, can access authorization information in Splunk Web. To view the roles assigned to a user, select Settings > Access controls and click Users. To determine the capabilities assigned to a role, select Settings > Access controls and click Roles.

App and user context

Typically, knowledge objects, such as saved searches or event types, have an app/user context that is the namespace. For more information about specifying a namespace, see Namespace in the REST API User Manual.

Splunk Cloud Platform limitations

As a Splunk Cloud Platform user, you are restricted to interacting with the search tier only with the REST API.

Some but not all KV store endpoints are available in Splunk Cloud Platform. Instead, authorized users can access and configure the KV store lookup definitions in the Splunk Cloud Platform user interface. KV store collections, however, cannot be modified with the user interface.

See Access requirements and limitations for the Splunk Cloud Platform REST API in the the REST API Tutorials manual for more information.

App KV Store REST API features

The app KV Store provides the following functions.

Create, read, list and delete EAI operations on a collection.
Create, read, list and delete indexes for a given collection. All collection methods are EAI-compatible. These are not regular EAI operations because the indexes are stored as attributes on the EAI entity. Users must have collection permission to use these operations for a particular collection.
The ability to insert, read, update and delete a single key and key value, for a given collection, provided that you have permission to modify the collection.
- Duplicate keys are not allowed.
- All updates are wholesale updates.
- Partial value updates are not available.
The ability to query key-value pairs, for a given collection. For example, to find all the documents that have a host field with a value, use the query
```
{"host": "bar.com"}
```
.

Only the following operations are permitted.

$gt
$gte
$lt
$lte
$ne
$and
$or,
$not.

The API also supports mechanisms for projection, sorting and pagination.

Batch reads, inserts, and/or updates for a given collection. This supports high-volume operations and limits network traffic. These operations have the same semantics as their individual counterparts.

Limits

16MB per record
1KB per accelerated field

Collections

A Collection is a container of "documents" or "values". Most documents have a similar structure, although structure is not enforced. For example, a Collection might have Notable Events or entities.

Collections are declared in a collections.conf configuration file, where a stanza is a single collection.

Collection permissioning

Collections are Splunk entities, which means they are permissioned using the Splunk RBAC/ACL system. Only app-level permissions are available for collections. This means that collections can be defined only in the etc/apps/<app>/<default|local> directory, not in etc/users/.... Attempts to create a user-specific Collection using the API fail and Collections created manually in the .conf files are ignored.

Data

A collection stores key-value pairs consisting of a user-specified or autogenerated key and a JSON document-formatted value. A JSON document/object of values is enforced for simplicity instead of using integer or string raw values. For example, {"_key": "10.0.0.1", host: "bar.com"} defines the key ("10.0.0.1") and the full JSON document value.

A key is fetched, updated, and deleted. Partial updates, including array append, are not available. You must set the whole document. Keys are unique within a collection and support only the basic JSON data types: boolean, string, number, object, array, and null.

A value is specified in the _key attribute of the document and, if one is not specified, a random value with a desirable sort order is automatically generated.

A collection is defined in the following directory location.

/servicesNS/nobody/search/storage/collections/mycollection

You can use the following URLs to access the collection.

/servicesNS/nobody/search/storage/collections/mycollection
/servicesNS/itay/search/storage/collections/mycollection
/servicesNS/mark/search/storage/collections/mycollection

Namespaces are separate and data are not interleaved. Wildcard operations on the <user> or <app> parts of the namespace are not available.

User-specific collections are not supported. All users, including nobody, share a single namespace and _key is unique across all namespaces.

Types

You can define a set of types for some fields in a collection. If a field has no type defined, it uses the type submitted over the API. If a type conversion fails, the insert/update operation is aborted.

Here is an example collection in collections.conf.

[mycollection]
field.enabled = bool
field.data.range = cidr

The data inserted or updated in this collection have the fields enabled and range-converted to the applicable type.

* array
* number
* boolean
* time
* string
* cidr

Items need to be specified using the field prefix only if type needs to be enforced for the field. The array type is not enforceable so does not need to be specified.

Arrays

Arrays do not change a nested key name. For example, the key "a.b.c" can refer to {"a": {"b": {"c": 1}}} or {"a": {"b": [{"c": 1}]}}.

Numbers

All strings/numbers/booleans are converted to doubles. If a number fails to convert, it inserts the string instead.

Booleans

All numbers/strings/booleans are converted to booleans. If a boolean fails to convert, it inserts the string instead. Epoch time can be a string or a number. If it fails to convert, the string version is inserted.

Strings

Everything is converted to a string.

CIDR

All provided strings are converted to canonical CIDR strings. Here is an example.

127.0.0.1       -> 127.000.000.001/32
127.0.0.0/24    -> 127.000.000.000/24
127.0.0/24      -> 127.000.000.000/24

Here is an example of similar canonicalization for IPv6.

2001:db8::/96               -> 2001:0db8:0000:0000:0000:0000:0000:0000/096
2001:db8:0:0:0:0:0:ffff     -> 2001:0db8:0000:0000:0000:0000:0000:ffff/128

Queries

Combining the Data and Index patterns gives the ability to do basic querying. Queries are limited to the basic operators available in most databases, and at the moment do not expose in-array and not-in-array containers. A query can have the following operators.

gt
gte
lt
lte
eq
neq
or
and
not

Queries are permitted regardless of whether an index covers the query. If it the index does not cover the query, the query takes longer to complete. For example, here is a query to find all Notable Events authored by userA or userB.

{ "$or": [ { "author": "userA" }, { "author": "userB" } ] }

Use $and to search for a range of values. For example, use the following query to find all events with a time field value between 12 and 13.

{ "$and": [ { "time": { "$gt": "12" } }, { "time": { "$lt": "13" } } ] }

REST methods

The REST API for App KV Store implements the GET, POST, and DELETE methods described in this reference.

Defining KV Store lookups

For information on using the REST API to define or update a KV Store lookup, see data/transforms/lookups/ and data/transforms/lookups/{name}.

kvstore/backup/create

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/backup/create

POST

Create a KV Store backup archive file.

Usage details

Request parameters

Name	Type	Description
archiveName	String	Specify a file name for the backup.
appName	String	Specify a target app for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
collectionName	String	Specify a target collection for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
pointInTime	Boolean	Defaults to false. To take a consistent backup, set it to true. Only available for single-instance deployments.
cancel	Boolean	Defaults to false. Set it to true to cancel an in-progress backup. Only available if pointInTime is set to true.
parallelCollections	Number	Defaults to 1. Raise the number to increase the number of collections to back up in parallel. Only available if pointInTime is set to true.

Returned values

Name	Description
status	Code `200` for success, and code `404` for failure.

Example request and response

XML request

curl -k -u admin:changed -X POST https://localhost:8089/services/kvstore/backup/create -d 'archiveName=sampleArchive&appName=search&collectionName=testcollection'

XML response

  <title>kvstorebackup</title>
  <id>https://localhost:8089/services/kvstore/backup</id>
  <updated>2018-04-05T14:18:14-07:00</updated>
  <generator build="dbd2996dbabd8f6751329c845684e0ffc9d1cca3" version="20180302"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/backup/_acl" rel="_acl"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>

kvstore/backup/restore

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/backup/restore

POST

Extracts the KV Store backup archive file and restores the KV Store.

Usage details

Request parameters

Name	Type	Description
archiveName	String	Required. Specifies the name of the backup file.
appName	String	Specify a target app for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
collectionName	String	Specify a target collection for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
pointInTime	Boolean	Defaults to false. To restore from a backup taken with consistency, set it to true.
cancel	Boolean	Defaults to false. Set it to true to cancel an in-progress restore. Only available if pointInTime set to true.
parallelCollections	Number	Defaults to 1. Raise the number to increase the number of collections to restore in parallel, which speeds up the store. Only available if pointInTime set to true.
insertionsWorkersPerCollection	Number	Defaults to 1. Raise to increase the number of insertion workers per collection, which speeds up the restore. Only available if pointInTime set to true.

Returned values

Name	Description
status	Code `200` for success, and code `404` for failure.

Example request and response

XML request

curl -k -u admin:changed -X POST https://localhost:8089/services/kvstore/backup/restore -d 'archiveName=kvdump.tar.gz&appName=search&collectionName=testcollection'

XML response

  <title>kvstorebackup</title>
  <id>https://localhost:8089/services/kvstore/backup</id>
  <updated>2018-04-05T14:10:00-07:00</updated>
  <generator build="dbd2996dbabd8f6751329c845684e0ffc9d1cca3" version="20180302"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/backup/_acl" rel="_acl"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>

kvstore/control/maintenance

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/control/maintenance

Access KV store maintenance mode for standalone deployments.

POST

Toggle maintenance mode.

Request parameters

Name	Type	Description
mode	Boolean	Required. Type true to enter maintenance mode. To exit, type false.

Example request and response

XML Request

curl -ku admin:changeme -X POST https://localhost:8089/services/kvstore/control/maintenance -d 'mode=false'

XML Response

<title>kvstorecontrol</title>
  <id>https://localhost:8089/services/kvstore/control</id>
  <updated>2021-03-29T14:53:25-07:00</updated>
  <generator build="c470713bc601c2961f6dba368c5b4c6628687211" version="20210304"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/control/maintenance" rel="maintenance"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>

kvstore/status

https://<host>:<mPort>/services/kvstore/status

Access KV store status information for standalone or search head clustering (SHC) deployments. For SHC deployments, provides information on SHC members where KV Store is enabled and used for replication.

See also the following KV Store introspection endpoints.

GET

Access KV store status information.

Usage details

Request parameters
None.

Returned values

Name	Description
current	Includes the following indicators for the machine making the GET request. `backupRestoreStatus`: Status for a KV Store backup in progress. One of the following values: `Busy`: Backup or restore in progress. `Failed`: Restore failed to extract an archive file. `Ready`: Ready to run a backup or restore. `Shutdown`: KV Store is in the process of shutting down. `backupRestoreProgress`: Counter that increments each time a collection finishes being backed up or restored. `date`: DateTime when this status was retrieved. `datesec`: Unix timestamp, equivalent to `date` `disabled`: If KV Store is disabled on the current member. `0` means enabled, `1` means disabled. `guid`: Instance ID of the current member. `oplogEndTimestamp`: Last recorded timestamp in the operations log. Last update in all KV Store collections. Compare this indicator to other instances to check if KV Store members are not up to date. `oplogEndTimestampSec`: Unix timestamp equivalent to `oplogEndTimestamp` `oplogStartTimestamp`: First recorded timestamp in the operations log. `oplogStartTimestampSec`: Unix timestamp equivalent to `oplogStartTimestamp` `port`: KV Store port `replicaSet`: Replica set name. Instance ID by default for standalone mode. Configured in `server.conf` for SHC. `replicationStatus`: In standalone mode, this is `KV Store captain`. Otherwise, one of the following values. `Startup` `KV Store captain` `Non-captain KV Store member` `Recovering` `Initial Sync` `Unknown status` `Down` `Rollback` `Removed` `standalone`: Indicates whether the machine making the request is a standalone member or SHC member. `1` indicates a standalone member. `status`: KV Store status. One of the following values. `unknown` `disabled` `starting` `ready` `failed` `shuttingdown`
Enabled KV Store members	Returned for SHC deployments. Lists the following values for SHC members where KV Store is enabled and used for replication. `guid`: Instance ID of the current member. `hostAndPort`: Address used for replication between KV Store members and for accessing members from `splunkd`. Can be configured in `server.conf`.
members	For KV Store members, lists the following indicators. `configVersion`: Version number that increases each time the KV Store cluster is updated. `electionDate`: DateTime for election. `electionDateSec`: Unix equivalent of `electionDate` `hostAndPort`: Address used for replication between KV Store members and for accessing members from `splunkd`. Can be configured in `server.conf`. `lastHeartbeat`: Last time the requesting member sent a heartbeat to this member. `lastHeartbeatRecv`: Last time this member replied on heartbeat. `lastHeartbeatRecvSec`: Unix equivalent of `lastHeartbeatRecv` `lastHeartbeatSec`: Unix equivalent of `lastHeartbeat` `optimeDate`: Last recorded timestamp for this member in the operations log. `optimeDateSec`: Unix equivalent of `optimeDate` `pingMs`: Latency (milliseconds for round trip. `replicationStatus`: In standalone mode, this is `KV Store captain`. Otherwise, one of the following values. `Startup` `KV Store captain` `Non-captain KV Store member` `Recovering` `Initial Sync` `Unknown status` `Down` `Rollback` `Removed` `uptime`: Number of seconds this member has been online.

Example request and response

XML request

curl -k -u admin:changed https://localhost:8089/services/kvstore/status

XML response

  <title>kvstorestatus</title>
  <id>https://localhost:8089/services/kvstore/status</id>
  <updated>2016-09-09T15:04:35-07:00</updated>
  <generator build="c9afb322d148" version="6.5.0"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/status/_acl" rel="_acl"/>
  <opensearch:totalResults>1</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>
  <entry>
    <title>status</title>
    <id>https://localhost:8089/services/kvstore/status/status</id>
    <updated>2016-09-09T15:04:35-07:00</updated>
    <link href="/services/kvstore/status/status" rel="alternate"/>
    <author>
      <name>system</name>
    </author>
    <link href="/services/kvstore/status/status" rel="list"/>
    <content type="text/xml">
      <s:dict>
        <s:key name="current">
          <s:dict>
            <s:key name="backupRestoreStatus">Ready</s:key>
            <s:key name="date">Fri Sep  9 15:04:35 2016</s:key>
            <s:key name="dateSec">1473458675.554000</s:key>
            <s:key name="disabled">0</s:key>
            <s:key name="guid">B4E173F1-02AA-4D46-9DDC-ECF0016F165E</s:key>
            <s:key name="oplogEndTimestamp">Fri Sep  9 14:17:01 2016</s:key>
            <s:key name="oplogEndTimestampSec">1473455821.000000</s:key>
            <s:key name="oplogStartTimestamp">Fri Sep  9 07:16:53 2016</s:key>
            <s:key name="oplogStartTimestampSec">1473430613.000000</s:key>
            <s:key name="port">8191</s:key>
            <s:key name="replicaSet">B4E173F1-02AA-4D46-9DDC-ECF0016F165E</s:key>
            <s:key name="replicationStatus">KV store captain</s:key>
            <s:key name="standalone">1</s:key>
            <s:key name="status">ready</s:key>
          </s:dict>
        </s:key>
        <s:key name="eai:acl">
          <s:dict>
            <s:key name="app"></s:key>
            <s:key name="can_list">1</s:key>
            <s:key name="can_write">1</s:key>
            <s:key name="modifiable">0</s:key>
            <s:key name="owner">system</s:key>
            <s:key name="perms">
              <s:dict>
                <s:key name="read">
                  <s:list>
                    <s:item>admin</s:item>
                    <s:item>splunk-system-role</s:item>
                  </s:list>
                </s:key>
                <s:key name="write">
                  <s:list>
                    <s:item>admin</s:item>
                    <s:item>splunk-system-role</s:item>
                  </s:list>
                </s:key>
              </s:dict>
            </s:key>
            <s:key name="removable">0</s:key>
            <s:key name="sharing">system</s:key>
          </s:dict>
        </s:key>
        <s:key name="members">
          <s:dict>
            <s:key name="0">
              <s:dict>
                <s:key name="configVersion">1</s:key>
                <s:key name="electionDate">Fri Sep  9 07:16:53 2016</s:key>
                <s:key name="electionDateSec">1473430613.000000</s:key>
                <s:key name="hostAndPort">127.0.0.1:8191</s:key>
                <s:key name="lastHeartbeat"></s:key>
                <s:key name="lastHeartbeatRecv"></s:key>
                <s:key name="lastHeartbeatRecvSec"></s:key>
                <s:key name="lastHeartbeatSec"></s:key>
                <s:key name="optimeDate">Fri Sep  9 14:17:01 2016</s:key>
                <s:key name="optimeDateSec">1473455821.000000</s:key>
                <s:key name="pingMs"></s:key>
                <s:key name="replicationStatus">KV store captain</s:key>
                <s:key name="uptime">28071</s:key>
              </s:dict>
            </s:key>
          </s:dict>
        </s:key>
      </s:dict>
    </content>
  </entry>

shcluster/captain/kvmigrate/start

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://<host>:<mPort>/services/shcluster/captain/kvmigrate/start

Start migration of the KV store storage engine.

POST

Start the migration.

Request parameters

Name	Type	Description
storageEngine	String	Required. Name of target storage engine, wiredTiger or mmap.
isDryRun	Boolean	Type true to complete pre-flight checks and exit without migrating. Setting is false by default.
maxRetries	Number	Number of times to retry a failed migration, per member.
clusterPerc	Number	Percentage of peers to migrate.
peersList	String	Names of peers to migrate, listed with name and management port. For example: peersList="server1:8089,server2:8089,server3:8089"

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8099/services/shcluster/captain/kvmigrate/start -X POST -d storageEngine=wiredTiger -d clusterPerc=50

XML Response

.  <s:messages>                                                                                                                          
    <s:msg type="INFO">SHC KV Store migration has been successfully triggered</s:msg>                                                   
  </s:messages>

shcluster/captain/kvmigrate/status

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://localhost:8099/services/shcluster/captain/kvmigrate/status

Check the status of a KV store storage engine migration.

GET

Check the status.

Request parameters
None.

Returned values

Name	Description
clusterPerc	Percentage of cluster members that have completed migration.
migrationID	ID number for the migration.
migrationStartTime	Timestamp that the migration began.
peerRetryCount	Number of times that the peer failed to migrate and retried.
status	Status of the migration.
storageEngine	Target storage engine.

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8099/services/shcluster/captain/kvmigrate/status

XML Response

...
        <s:key name="general">
          <s:dict>
            <s:key name="clusterPerc">50</s:key>
            <s:key name="migrationID">1596760435</s:key>
            <s:key name="migrationStartTime">Thu Aug  6 17:33:55 2020</s:key>
            <s:key name="peerRetryCount">1</s:key>
            <s:key name="status">kvstore_migration_started</s:key>
            <s:key name="storageEngine">wiredTiger</s:key>
          </s:dict>
        </s:key>
        <s:key name="peerStatus">
          <s:dict>
            <s:key name="https://fool02:8099">
              <s:dict>
                <s:key name="attempt_number">1</s:key>
                <s:key name="migrationStatus">migration_in_progress</s:key>
              </s:dict>
            </s:key>
            <s:key name="https://fool02:8103">
              <s:dict>
                <s:key name="attempt_number">1</s:key>
                <s:key name="migrationStatus">migration_succeeded</s:key>
              </s:dict>
            </s:key>

shcluster/captain/kvmigrate/stop

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://<host>:<mPort>/services/shcluster/captain/kvmigrate/stop

Stop the migration of your KV store storage engine.

POST

Stop the migration.

Request parameters
None.

Example request and response

XML Request

curl -k -u admin:changeme -X POST https://localhost:8099/services/shcluster/captain/kvmigrate/stop

XML Response

...
  <s:messages>                                                                                                                          
    <s:msg type="INFO">KV Store migration stop initiated</s:msg>                                                                        
  </s:messages>

storage/collections/config

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config

Access and create collections.

GET

List all collections.

Request parameters
None.

Returned values

Name	Description
disabled	Boolean indicating collection state. By default, the value is `false`, indicating that the collection is enabled.
profilingEnabled	Boolean indicating profiling status of slow-running operations. By default, this value is `false`, meaning that profiling is disabled.
profilingThresholdMs	Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is `true`.

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:28:27-07:00</updated> <generator build="229629" version="6.2"/>

<author>
  <name>Splunk</name>
</author>
<link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
<link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
... opensearch nodes elided ...
<s:messages/>
<entry>
  <title>kvstoredemo</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/kvstoredemo</id>
  <updated>2014-09-02T11:28:27-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ... </s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>
<entry>
  <title>test</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
  <updated>2014-09-02T11:28:27-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ...</s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>

POST

Create a collection.

Request parameters

Name	Type	Description
name	String	Required. Collection name
profilingEnabled	Boolean	A `collections.conf` file property that affects profilingThresholdMs. Defaults to `false`. Enable profiling of slow-running operations by setting profilingEnabled to `true`.
profilingThresholdMs	Number	Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is `true`. Defaults to `100`. Set to `0` to log all slow-running operations.

Returned values

Name	Description
disabled	Boolean indicating collection state. By default, the value is `false`, indicating that the collection is enabled.
profilingEnabled	Profiling status of slow-running operations, profilingThresholdMs. Defaults to `false`.
profilingThresholdMs	Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is `true`.

Example request and response

XML Request

curl -k -u admin:changeme -d name=test1 https://localhost:8089/servicesNS/nobody/search/storage/collections/config

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:25:32-07:00</updated> <generator build="229629" version="6.2"/>

<author>
  <name>Splunk</name>
</author>
<link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
<link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
... opensearch nodes elided ...
<s:messages/>
<entry>
  <title>test1</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test1</id>
  <updated>2014-09-02T11:25:32-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ... </s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>

storage/collections/config/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config/{collection}

Access, delete, or update a specific {collection}.

DELETE

Delete a specific {collection}.

Request parameters
None

Returned values
None

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test -X DELETE

XML Response

...
 <title>collections-conf</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id>
 <updated>2014-06-02T20:08:11-07:00</updated>
 <generator build="210538" version="20140530"/>
 <author>
   <name>Splunk</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
 ... opensearch nodes elided ...
 <s:messages/>

GET

Access a specific {collection}.

Request parameters
None

Returned values

Name	Description
disabled	Boolean indicating collection state. By default, the value is `0`, meaning that the collection is enabled.
field.<fieldName>	Field type. One of the following values. `array` `number` `bool` `string` `cidr` `time`
accelerated_fields.<field_name>	Field acceleration name and JSON definition.
enforceTypes	Boolean indicating if data types are enforced when inserting data into the collection. Defaults to `false`.
profilingEnabled	Profiling status of slow-running operations, affecting profilingThresholdMs. By default, the value is `false`, meaning that profiling is disabled. If `true`, profiling is enabled.
profilingThresholdMs	Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is `true`.
replicate	Boolean indicating whether the collection is replicated on indexers. Defaults to `false`, meaning that this collection is not replicated, and lookups that depend on the collection will not be available. However, if you run a lookup command with `local=true`, local lookups will still be available. When `true`, this collection is replicated on indexers.
replication_dump_maximum_file_size	Indicates the maximum file size (in KB) for each dump file when replication_dump_strategy=`auto`. Defaults to 10240KB. If this value is larger than `concerningReplicatedFileSize`, which is set in the `distsearch.conf` file, the value of `concerningReplicatedFileSize` is used instead. KV Store does not pre-calculate the size of the records that will be written to disk, so the size of the resulting files can be affected by the `max_rows_in_memory_per_dump` setting in the `limits.conf` file.
replication_dump_strategy	One of the following two values. `auto`: Default. Dumps are stored in multiple files when the size of the collection exceeds the value of replication_dump_maximum_file_size. `one_file`: Dump files are stored in a single file.

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:42:54-07:00</updated> <generator build="229629" version="6.2"/>

<author>

 <name>Splunk</name>
</author> <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/> <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/> ... opensearch nodes elided ... <s:messages/> <entry>

 <title>test</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
 <updated>2014-09-02T11:42:54-07:00</updated>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
 <author>
   <name>admin</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
 <content type="text/xml">
   <s:dict>
     <s:key name="disabled">0</s:key>
     <s:key name="eai:acl"> ... elided ... </s:key>
     <s:key name="eai:appName">search</s:key>
     <s:key name="eai:attributes"> ... elided ... </s:key>
     <s:key name="eai:userName">nobody</s:key>
     <s:key name="profilingEnabled">false</s:key>
     <s:key name="profilingThresholdMs">100</s:key>
   </s:dict>
 </content>
</entry>

POST

Update a specific {collection}.

Request parameters

Name	Type	Description
field.<fieldName>	String	Field type. One of the following values. `array` `number` `bool` `string` `cidr` `time`
accelerated_fields.<field_name>	String, JSON (see description)	The name of a field acceleration (string) and its definition, in JSON key value format. For example, `accelerated_fields.my_accel = {"id": 1}`

Returned values

Name	Description
disabled	Collection state: `true` = disabled. `false` = [Default] enabled.
field.<fieldName>	Field type. One of the following values. `array` `number` `bool` `string` `cidr` `time`
accelerated_fields.<field_name>	The name of a field acceleration (string) and its definition, in JSON key value format. For example, `accelerated_fields.my_accel = {"id": 1}`
profilingEnabled	Profiling status of slow-running operations, affecting profilingThresholdMs. By default, the value is `false`, meaning that profiling is disabled. If `true`, profiling is enabled.
profilingThresholdMs	Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is `true`.
replicate	Boolean indicating whether the collection is replicated on indexers. Defaults to `false`, meaning that this collection is not replicated, and lookups that depend on the collection will not be available. However, if you run a lookup command with `local=true`, local lookups will still be available. When `true`, this collection is replicated on indexers.
replication_dump_maximum_file_size	Indicates the maximum file size (in KB) for each dump file when replication_dump_strategy=`auto`. Defaults to 10240KB. If this value is larger than `concerningReplicatedFileSize`, which is set in the `distsearch.conf` file, the value of `concerningReplicatedFileSize` is used instead. KV Store does not pre-calculate the size of the records that will be written to disk, so the size of the resulting files can be affected by the `max_rows_in_memory_per_dump` setting in the `limits.conf` file.
replication_dump_strategy	One of the following two values. `auto`: Default. Dumps are stored in multiple files when the size of the collection exceeds the value of replication_dump_maximum_file_size. `one_file`: Dump files are stored in a single file.

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test -d 'accelerated_fields.foo={"a": 1}' -d 'accelerated_fields.bar={"b": -1}' -d "field.a=number" -d "field.b=cidr"

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:50:57-07:00</updated> <generator build="229629" version="6.2"/>

<author>

 <name>Splunk</name>
</author> <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/> <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/> ... opensearch nodes elided ... <s:messages/> <entry>

 <title>test</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
 <updated>2014-09-02T11:50:57-07:00</updated>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
 <author>
   <name>admin</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
 <content type="text/xml">
   <s:dict>
     <s:key name="disabled">0</s:key>
     <s:key name="eai:acl"> ... elided ... </s:key>
     <s:key name="eai:appName">search</s:key>
     <s:key name="eai:userName">nobody</s:key>
     <s:key name="accelerated_fields.foo">{"a": 1}
accelerated_fields.bar={"b": -1} field.a=number field.b=cidr</s:key>

     <s:key name="profilingEnabled">false</s:key>
     <s:key name="profilingThresholdMs">100</s:key>
   </s:dict>
 </content>
</entry>

storage/collections/data/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}

Access and manage items of a collection.

DELETE

Delete items in the {collection} or delete an entire collection.

Usage details
Use the query parameter to specify which entries to delete. If no query is provided, all entries in the collection are deleted.

Request parameters

Name	Type	Description
query	JSON object	Query JSON object. Conditional operators: `$gt`, `$gte`, `$lt`, `$lte`, and `$ne` Logical operators: `$and`, `$or`, and ,`$not` (invert conditional operators) Examples: `query={"title":"Item"}` (Select all documents with property `title` that has value `Item`) `query={"price":{"$gt":5}}` (Select all documents with `price` greater than `5`)

Returned values
None

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA -X DELETE

XML Response
No values returned.

GET

Access a specific {collection}.

Request parameters

Parameter order in the request does not matter. Parameters are always applied in the order of sort, skip, and limit.

Name	Type	Description
fields	String	Comma-separated list of fields to include (`1`) or exclude (`0`). A fields value cannot contain both include and exclude specifications except for exclusion of the `_key` field. Examples: `fields=firstname,surname` (Include only `firstname`, `surname`, and `_key` fields) `fields=firstname,surname,_key:0` (Include only the `firstname` and `surname` fields) `fields=address:0` (Include all fields except the `address` field)
shared	Boolean	Defaults to false. Set to true to return records for the specified user as well as records for the `nobody` user.
limit	Number	Maximum number of items to return. For example, to return five items, use `limit=5`.
skip	Number	Number of items to skip from the start. For example, to skip the first ten items, use `skip=10`.
sort	String	Sort order. Examples: `sort=surname` (Sort by `surname`, ascending) `sort=surname,firstname` (Sort by `surname`, ascending, after `firstname`, ascending) `sort=surname:-1,firstname:1` (Sort by `surname`, descending, after `firstname`, ascending `sort=surname:1,first name` (Sort by `surname`, ascending, after `firstname`, ascending
query	JSON object	Query JSON object. Conditional operators: `$gt`, `$gte`, `$lt`, `$lte`, and `$ne` Logical operators: `$and`, `$or`, and ,`$not` (invert conditional operators) Examples: `query={"title":"Item"}` (Select all documents with property `title` that has value `Item`) `query={"price":{"$gt":5}}` (Select all documents with `price` greater than `5`)

Returned values
The response includes a JSON document.

Example

[ { "myKey" : "abc", "_user" : "nobody", "_key" : "5410be5441ba15298e4624d1" } ]

POST

Insert an item into the {collection}.

Usage details
The Content-Type header must be application/json.

The _key is autogenerated, if not manually specified.

Request parameters
Make sure that the item to add is a JSON-formatted document, such as the following example.

{ "name": "A" }

Returned values
The response includes a JSON-formatted document key, such as the following example.

{
    "_key": "530bc62fc9a6577fdf13651f"
}

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA -H "Content-Type: application/json" -d '{ "myKey": "abc",  "myOtherKey": "abcdef"}'

XML Response

{"_key":"5410be5441ba15298e4624d1"}

storage/collections/data/{collection}/{key}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}/{key}

Access and manage a specific {key} item in a {collection}.

DELETE

Delete a collection item.

Request parameters
None

Returned values
None

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/5410caf041ba15298e4624d6 -X DELETE

XML Response
No values returned.

GET

Access a collection item.

Request parameters
None

Returned values
The response includes a JSON-formatted {key} document, such as the following example.

{"myKey" : "abc", "myOtherKey" : "uvwxyz", "_user" : "nobody", "_key" : "5410c8dc41ba15298e4624d5"}

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/5410c8dc41ba15298e4624d5

XML Response

{ "myKey" : "abc", "myOtherKey" : "uvwxyz", "_user" : "nobody", "_key" : "5410c8dc41ba15298e4624d5" }

POST

Update a collection item.

Usage details
The header Content-Type must be application/json.

Request parameters
Pass in a JSON-formatted document for the {key} that you are updating. For example, use { "myKey": "fizz"}.

Returned values
The response includes a JSON-formatted {key} such as the following example.

{"_key":"5410c8dc41ba15298e4624d5"}

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/5410c8dc41ba15298e4624d5 -H "Content-Type: application/json" -d '{ "myKey": "fizz"}'

XML Response

{"_key":"5410c8dc41ba15298e4624d5"}

storage/collections/data/{collection}/batch_find

https://<host>:<mPort>/servicesNS/storage/collections/data/{collection}/batch_find

Perform multiple queries in a batch.

POST

Perform multiple queries in a batch.

Request parameters
Pass in the JSON array of queries to run. Parameter order in the request does not matter. Parameters are always applied in the order of sort, skip, and limit.

Name	Type	Description
fields	String	Comma-separated list of fields to include (`1`) or exclude (`0`). A fields value cannot contain both include and exclude specifications except for exclusion of the `_key` field. Examples: `fields=firstname,surname` (Include only `firstname`, `surname`, and `_key` fields) `fields=firstname,surname,_key:0` (Include only the `firstname` and `surname` fields. `fields=address:0` (Include all fields except the `address` field)
shared	Boolean	Defaults to false. Set to true to return records for the specified user as well as records for the `nobody` user.
limit	Number	Maximum number of items to return. For example, to return five items, use `limit=5`.
skip	Number	Number of items to skip from the start. For example, to skip the first ten items, use `skip=10`.
sort	String	Sort order. Examples: `sort=surname` (Sort by `surname`, ascending) `sort=surname,firstname` (Sort by `surname`, ascending, after `firstname`, ascending) `sort=surname:-1,firstname:1` (Sort by `surname`, descending, after `firstname`, ascending `sort=surname:1,first name` (Sort by `surname`, ascending, after `firstname`, ascending
query	JSON object	Query JSON object. Conditional operators: `$gt`, `$gte`, `$lt`, `$lte`, and `$ne` Logical operators: `$and`, `$or`, and ,`$not` (invert conditional operators) Examples: `query={"title":"Item"}` (Select all documents with property `title` that has value `Item`) `query={"price":{"$gt":5}}` (Select all documents with `price` greater than `5`)

Returned values
The response includes a JSON-formatted array of an array of records. The first array of records is the result of the first query, the second array of records is the result of the second query, and so on.

Example request and response

XML Request

queries='['
queries+='{"query": {"myKey": "def"}},'
queries+='{"query": {"myKey": "abc"}},'
queries+='{"query": {"myKey": "jkl"}},'
queries+='{"shared": true, "fields": {"myKey": 1}, "sort": [{"myKey": 1}], "limit": 2}'
queries+=']'
curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/batch_find -H "Content-Type: application/json" -d "$queries"

XML Response

[
  [{"myKey":"def","_user":"nobody","_key":"f3jel0421dsw1024f3fd1a78"}],
  [{"myKey":"abc","_user":"nobody","_key":"b2jcw0405saw1221177aa78f"}],
  [{"myKey":"jkl","_user":"nobody","_key":"3b4b0d9ef63cc83959e5ed77"}],
  [{"myKey":"abc"},{"myKey":"def"}]
]

storage/collections/data/{collection}/batch_save

https://<host>:<mPort>/servicesNS/storage/collections/data/{collection}/batch_save

Perform multiple save operations in a batch.

POST

Perform multiple save operations in a batch.

Usage details

The request header Content-Type must be application/json.

If a document does not have a _key field or store does not have a matching document ID, a _key new document is created.

If an operation fails, the server stops batch processing and returns an HTTP status code other than 200.

Request parameters
Pass in the JSON array of documents to save.

Returned values
The response includes a JSON-formatted list of keys added, such as the following example.

[ "5410c43241ba15298e4624d3", "5418c54e41ba152267763c01" ]

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/batch_save -H "Content-Type: application/json" -d '[{ "_key": "5410c43241ba15298e4624d3", "name": "AAAAAAAA" },{ "name": "A" }]'

XML Response

[ "5410c43241ba15298e4624d3", "5418c54e41ba152267763c01" ]

Related answers from Splunk Community

KV store endpoint descriptions

REST API usage details

Review ACL information for an endpoint

Authentication and Authorization

App and user context

Splunk Cloud Platform limitations

App KV Store REST API features

Limits

Collections

Collection permissioning

Data

Types

Arrays

Numbers

Booleans

Strings

CIDR

Queries

REST methods

Defining KV Store lookups

kvstore/backup/create

POST

kvstore/backup/restore

POST

kvstore/control/maintenance

POST

kvstore/status

GET

shcluster/captain/kvmigrate/start

POST

shcluster/captain/kvmigrate/status

GET

shcluster/captain/kvmigrate/stop

POST

storage/collections/config

GET

POST

storage/collections/config/{collection}

DELETE

GET

POST

storage/collections/data/{collection}

DELETE

GET

POST

storage/collections/data/{collection}/{key}

DELETE

GET

POST

storage/collections/data/{collection}/batch_find

POST

storage/collections/data/{collection}/batch_save

POST

Comments

KV store endpoint descriptions