Splunk® Enterprise

REST API Reference Manual

KV store endpoint descriptions

This introduction describes syntax common to all app KV Store REST endpoints. For more information about the KV Store, see App Key Value Store on the Splunk developer portal.

REST API usage details

Review ACL information for an endpoint

To check Access Control List (ACL) properties for an endpoint, append /acl to the path. For more information see Access Control List in the REST API User Manual.

Authentication and Authorization

Username and password authentication is required for access to endpoints and REST operations.

Splunk users must have role and/or capability-based authorization to use REST endpoints. Users with an administrative role, such as admin, can access authorization information in Splunk Web. To view the roles assigned to a user, select Settings > Access controls and click Users. To determine the capabilities assigned to a role, select Settings > Access controls and click Roles.

App and user context

Typically, knowledge objects, such as saved searches or event types, have an app/user context that is the namespace. For more information about specifying a namespace, see Namespace in the REST API User Manual.

Splunk Cloud Platform limitations

As a Splunk Cloud Platform user, you are restricted to interacting with the search tier only with the REST API.

Some but not all KV store endpoints are available in Splunk Cloud Platform. Instead, authorized users can access and configure the KV store lookup definitions in the Splunk Cloud Platform user interface. KV store collections, however, cannot be modified with the user interface.

See Access requirements and limitations for the Splunk Cloud Platform REST API in the the REST API Tutorials manual for more information.

App KV Store REST API features

The app KV Store provides the following functions.

  • Create, read, list and delete EAI operations on a collection.
  • Create, read, list and delete indexes for a given collection. All collection methods are EAI-compatible. These are not regular EAI operations because the indexes are stored as attributes on the EAI entity. Users must have collection permission to use these operations for a particular collection.
  • The ability to insert, read, update and delete a single key and key value, for a given collection, provided that you have permission to modify the collection.
    • Duplicate keys are not allowed.
    • All updates are wholesale updates.
    • Partial value updates are not available.
  • The ability to query key-value pairs, for a given collection. For example, to find all the documents that have a host field with a value, use the query 
    {"host": "bar.com"}
    .

Only the following operations are permitted.

  • $gt
  • $gte
  • $lt
  • $lte
  • $ne
  • $and
  • $or,
  • $not.

The API also supports mechanisms for projection, sorting and pagination.

  • Batch reads, inserts, and/or updates for a given collection. This supports high-volume operations and limits network traffic. These operations have the same semantics as their individual counterparts.

Limits

  • 16MB per record
  • 1KB per accelerated field

Collections

A Collection is a container of "documents" or "values". Most documents have a similar structure, although structure is not enforced. For example, a Collection might have Notable Events or entities.

Collections are declared in a collections.conf configuration file, where a stanza is a single collection.

Collection permissioning

Collections are Splunk entities, which means they are permissioned using the Splunk RBAC/ACL system. Only app-level permissions are available for collections. This means that collections can be defined only in the etc/apps/<app>/<default|local> directory, not in etc/users/.... Attempts to create a user-specific Collection using the API fail and Collections created manually in the .conf files are ignored.

Data

A collection stores key-value pairs consisting of a user-specified or autogenerated key and a JSON document-formatted value. A JSON document/object of values is enforced for simplicity instead of using integer or string raw values. For example, {"_key": "10.0.0.1", host: "bar.com"} defines the key ("10.0.0.1") and the full JSON document value.

A key is fetched, updated, and deleted. Partial updates, including array append, are not available. You must set the whole document. Keys are unique within a collection and support only the basic JSON data types: boolean, string, number, object, array, and null.

A value is specified in the _key attribute of the document and, if one is not specified, a random value with a desirable sort order is automatically generated.

A collection is defined in the following directory location. 

/servicesNS/nobody/search/storage/collections/mycollection

You can use the following URLs to access the collection.

  • /servicesNS/nobody/search/storage/collections/mycollection
  • /servicesNS/itay/search/storage/collections/mycollection
  • /servicesNS/mark/search/storage/collections/mycollection

Namespaces are separate and data are not interleaved. Wildcard operations on the <user> or <app> parts of the namespace are not available.

User-specific collections are not supported. All users, including nobody, share a single namespace and _key is unique across all namespaces.

Types

You can define a set of types for some fields in a collection. If a field has no type defined, it uses the type submitted over the API. If a type conversion fails, the insert/update operation is aborted.

Here is an example collection in collections.conf. 

[mycollection]
field.enabled = bool
field.data.range = cidr

The data inserted or updated in this collection have the fields enabled and range-converted to the applicable type. 


* array
* number
* boolean
* time
* string
* cidr


Items need to be specified using the field prefix only if type needs to be enforced for the field. The array type is not enforceable so does not need to be specified.

Arrays

Arrays do not change a nested key name. For example, the key "a.b.c" can refer to {"a": {"b": {"c": 1}}} or {"a": {"b": [{"c": 1}]}}.

Numbers

All strings/numbers/booleans are converted to doubles. If a number fails to convert, it inserts the string instead.

Booleans

All numbers/strings/booleans are converted to booleans. If a boolean fails to convert, it inserts the string instead. Epoch time can be a string or a number. If it fails to convert, the string version is inserted.

Strings

Everything is converted to a string.

CIDR

All provided strings are converted to canonical CIDR strings. Here is an example. 

127.0.0.1       -> 127.000.000.001/32
127.0.0.0/24    -> 127.000.000.000/24
127.0.0/24      -> 127.000.000.000/24

Here is an example of similar canonicalization for IPv6. 

2001:db8::/96               -> 2001:0db8:0000:0000:0000:0000:0000:0000/096
2001:db8:0:0:0:0:0:ffff     -> 2001:0db8:0000:0000:0000:0000:0000:ffff/128

Queries

Combining the Data and Index patterns gives the ability to do basic querying. Queries are limited to the basic operators available in most databases, and at the moment do not expose in-array and not-in-array containers. A query can have the following operators.

  • gt
  • gte
  • lt
  • lte
  • eq
  • neq
  • or
  • and
  • not

Queries are permitted regardless of whether an index covers the query. If it the index does not cover the query, the query takes longer to complete. For example, here is a query to find all Notable Events authored by userA or userB. 

{ "$or": [ { "author": "userA" }, { "author": "userB" } ] }

Use $and to search for a range of values. For example, use the following query to find all events with a time field value between 12 and 13. 

{ "$and": [ { "time": { "$gt": "12" } }, { "time": { "$lt": "13" } } ] }

REST methods

The REST API for App KV Store implements the GET, POST, and DELETE methods described in this reference.


Defining KV Store lookups

For information on using the REST API to define or update a KV Store lookup, see data/transforms/lookups/ and data/transforms/lookups/{name}.

kvstore/backup/create

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/backup/create

POST

Create a KV Store backup archive file.

Usage details

Request parameters

Name Type Description
archiveName String Specify a file name for the backup.
appName String Specify a target app for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
collectionName String Specify a target collection for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
pointInTime Boolean Defaults to false. To take a consistent backup, set it to true. Only available for single-instance deployments.
cancel Boolean Defaults to false. Set it to true to cancel an in-progress backup. Only available if pointInTime is set to true.
parallelCollections Number Defaults to 1. Raise the number to increase the number of collections to back up in parallel. Only available if pointInTime is set to true.

Returned values

Name Description
status Code 200 for success, and code 404 for failure.


Example request and response

XML request

curl -k -u admin:changed -X POST https://localhost:8089/services/kvstore/backup/create -d 'archiveName=sampleArchive&appName=search&collectionName=testcollection'

XML response

  <title>kvstorebackup</title>
  <id>https://localhost:8089/services/kvstore/backup</id>
  <updated>2018-04-05T14:18:14-07:00</updated>
  <generator build="dbd2996dbabd8f6751329c845684e0ffc9d1cca3" version="20180302"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/backup/_acl" rel="_acl"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>

kvstore/backup/restore

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/backup/restore

POST

Extracts the KV Store backup archive file and restores the KV Store.

Usage details

Request parameters

Name Type Description
archiveName String Required. Specifies the name of the backup file.
appName String Specify a target app for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
collectionName String Specify a target collection for backup, rather than all of the KV Store. Only available if pointInTime is not set to true.
pointInTime Boolean Defaults to false. To restore from a backup taken with consistency, set it to true.
cancel Boolean Defaults to false. Set it to true to cancel an in-progress restore. Only available if pointInTime set to true.
parallelCollections Number Defaults to 1. Raise the number to increase the number of collections to restore in parallel, which speeds up the store. Only available if pointInTime set to true.
insertionsWorkersPerCollection Number Defaults to 1. Raise to increase the number of insertion workers per collection, which speeds up the restore. Only available if pointInTime set to true.

Returned values

Name Description
status Code 200 for success, and code 404 for failure.


Example request and response

XML request

curl -k -u admin:changed -X POST https://localhost:8089/services/kvstore/backup/restore -d 'archiveName=kvdump.tar.gz&appName=search&collectionName=testcollection'

XML response

  <title>kvstorebackup</title>
  <id>https://localhost:8089/services/kvstore/backup</id>
  <updated>2018-04-05T14:10:00-07:00</updated>
  <generator build="dbd2996dbabd8f6751329c845684e0ffc9d1cca3" version="20180302"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/backup/_acl" rel="_acl"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>

kvstore/control/maintenance

This endpoint is available only in Splunk Enterprise.

https://<host>:<mPort>/services/kvstore/control/maintenance

Access KV store maintenance mode for standalone deployments.

POST

Toggle maintenance mode.

Request parameters

Name Type Description
mode Boolean Required. Type true to enter maintenance mode. To exit, type false.

Example request and response


XML Request

curl -ku admin:changeme -X POST https://localhost:8089/services/kvstore/control/maintenance -d 'mode=false'

XML Response

<title>kvstorecontrol</title>
  <id>https://localhost:8089/services/kvstore/control</id>
  <updated>2021-03-29T14:53:25-07:00</updated>
  <generator build="c470713bc601c2961f6dba368c5b4c6628687211" version="20210304"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/control/maintenance" rel="maintenance"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>


kvstore/status

https://<host>:<mPort>/services/kvstore/status

Access KV store status information for standalone or search head clustering (SHC) deployments. For SHC deployments, provides information on SHC members where KV Store is enabled and used for replication.

See also the following KV Store introspection endpoints.


GET

Access KV store status information.

Usage details


Request parameters
None.

Returned values


Name Description
current Includes the following indicators for the machine making the GET request.
  • backupRestoreStatus: Status for a KV Store backup in progress. One of the following values:
    • Busy: Backup or restore in progress.
    • Failed: Restore failed to extract an archive file.
    • Ready: Ready to run a backup or restore.
    • Shutdown: KV Store is in the process of shutting down.
  • backupRestoreProgress: Counter that increments each time a collection finishes being backed up or restored.
  • date: DateTime when this status was retrieved.
  • datesec: Unix timestamp, equivalent to date
  • disabled: If KV Store is disabled on the current member. 0 means enabled, 1 means disabled.
  • guid: Instance ID of the current member.
  • oplogEndTimestamp: Last recorded timestamp in the operations log. Last update in all KV Store collections. Compare this indicator to other instances to check if KV Store members are not up to date.
  • oplogEndTimestampSec: Unix timestamp equivalent to oplogEndTimestamp
  • oplogStartTimestamp: First recorded timestamp in the operations log.
  • oplogStartTimestampSec: Unix timestamp equivalent to oplogStartTimestamp
  • port: KV Store port
  • replicaSet: Replica set name. Instance ID by default for standalone mode. Configured in server.conf for SHC.
  • replicationStatus: In standalone mode, this is KV Store captain. Otherwise, one of the following values.
    • Startup
    • KV Store captain
    • Non-captain KV Store member
    • Recovering
    • Initial Sync
    • Unknown status
    • Down
    • Rollback
    • Removed
  • standalone: Indicates whether the machine making the request is a standalone member or SHC member. 1 indicates a standalone member.
  • status: KV Store status. One of the following values.
    • unknown
    • disabled
    • starting
    • ready
    • failed
    • shuttingdown
Enabled KV Store members Returned for SHC deployments. Lists the following values for SHC members where KV Store is enabled and used for replication.
  • guid: Instance ID of the current member.
  • hostAndPort: Address used for replication between KV Store members and for accessing members from splunkd. Can be configured in server.conf.
members For KV Store members, lists the following indicators.
  • configVersion: Version number that increases each time the KV Store cluster is updated.
  • electionDate: DateTime for election.
  • electionDateSec: Unix equivalent of electionDate
  • hostAndPort: Address used for replication between KV Store members and for accessing members from splunkd. Can be configured in server.conf.
  • lastHeartbeat: Last time the requesting member sent a heartbeat to this member.
  • lastHeartbeatRecv: Last time this member replied on heartbeat.
  • lastHeartbeatRecvSec: Unix equivalent of lastHeartbeatRecv
  • lastHeartbeatSec: Unix equivalent of lastHeartbeat
  • optimeDate: Last recorded timestamp for this member in the operations log.
  • optimeDateSec: Unix equivalent of optimeDate
  • pingMs: Latency (milliseconds for round trip.
  • replicationStatus: In standalone mode, this is KV Store captain. Otherwise, one of the following values.
    • Startup
    • KV Store captain
    • Non-captain KV Store member
    • Recovering
    • Initial Sync
    • Unknown status
    • Down
    • Rollback
    • Removed
  • uptime: Number of seconds this member has been online.

Example request and response

XML request

curl -k -u admin:changed https://localhost:8089/services/kvstore/status

XML response

  <title>kvstorestatus</title>
  <id>https://localhost:8089/services/kvstore/status</id>
  <updated>2016-09-09T15:04:35-07:00</updated>
  <generator build="c9afb322d148" version="6.5.0"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/services/kvstore/status/_acl" rel="_acl"/>
  <opensearch:totalResults>1</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>
  <entry>
    <title>status</title>
    <id>https://localhost:8089/services/kvstore/status/status</id>
    <updated>2016-09-09T15:04:35-07:00</updated>
    <link href="/services/kvstore/status/status" rel="alternate"/>
    <author>
      <name>system</name>
    </author>
    <link href="/services/kvstore/status/status" rel="list"/>
    <content type="text/xml">
      <s:dict>
        <s:key name="current">
          <s:dict>
            <s:key name="backupRestoreStatus">Ready</s:key>
            <s:key name="date">Fri Sep  9 15:04:35 2016</s:key>
            <s:key name="dateSec">1473458675.554000</s:key>
            <s:key name="disabled">0</s:key>
            <s:key name="guid">B4E173F1-02AA-4D46-9DDC-ECF0016F165E</s:key>
            <s:key name="oplogEndTimestamp">Fri Sep  9 14:17:01 2016</s:key>
            <s:key name="oplogEndTimestampSec">1473455821.000000</s:key>
            <s:key name="oplogStartTimestamp">Fri Sep  9 07:16:53 2016</s:key>
            <s:key name="oplogStartTimestampSec">1473430613.000000</s:key>
            <s:key name="port">8191</s:key>
            <s:key name="replicaSet">B4E173F1-02AA-4D46-9DDC-ECF0016F165E</s:key>
            <s:key name="replicationStatus">KV store captain</s:key>
            <s:key name="standalone">1</s:key>
            <s:key name="status">ready</s:key>
          </s:dict>
        </s:key>
        <s:key name="eai:acl">
          <s:dict>
            <s:key name="app"></s:key>
            <s:key name="can_list">1</s:key>
            <s:key name="can_write">1</s:key>
            <s:key name="modifiable">0</s:key>
            <s:key name="owner">system</s:key>
            <s:key name="perms">
              <s:dict>
                <s:key name="read">
                  <s:list>
                    <s:item>admin</s:item>
                    <s:item>splunk-system-role</s:item>
                  </s:list>
                </s:key>
                <s:key name="write">
                  <s:list>
                    <s:item>admin</s:item>
                    <s:item>splunk-system-role</s:item>
                  </s:list>
                </s:key>
              </s:dict>
            </s:key>
            <s:key name="removable">0</s:key>
            <s:key name="sharing">system</s:key>
          </s:dict>
        </s:key>
        <s:key name="members">
          <s:dict>
            <s:key name="0">
              <s:dict>
                <s:key name="configVersion">1</s:key>
                <s:key name="electionDate">Fri Sep  9 07:16:53 2016</s:key>
                <s:key name="electionDateSec">1473430613.000000</s:key>
                <s:key name="hostAndPort">127.0.0.1:8191</s:key>
                <s:key name="lastHeartbeat"></s:key>
                <s:key name="lastHeartbeatRecv"></s:key>
                <s:key name="lastHeartbeatRecvSec"></s:key>
                <s:key name="lastHeartbeatSec"></s:key>
                <s:key name="optimeDate">Fri Sep  9 14:17:01 2016</s:key>
                <s:key name="optimeDateSec">1473455821.000000</s:key>
                <s:key name="pingMs"></s:key>
                <s:key name="replicationStatus">KV store captain</s:key>
                <s:key name="uptime">28071</s:key>
              </s:dict>
            </s:key>
          </s:dict>
        </s:key>
      </s:dict>
    </content>
  </entry>

shcluster/captain/kvmigrate/start

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://<host>:<mPort>/services/shcluster/captain/kvmigrate/start


Start migration of the KV store storage engine.


POST

Start the migration.

Request parameters

Name Type Description
storageEngine String Required. Name of target storage engine, wiredTiger or mmap.
isDryRun Boolean Type true to complete pre-flight checks and exit without migrating. Setting is false by default.
maxRetries Number Number of times to retry a failed migration, per member.
clusterPerc Number Percentage of peers to migrate.
peersList String Names of peers to migrate, listed with name and management port. For example: 
peersList="server1:8089,server2:8089,server3:8089"

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8099/services/shcluster/captain/kvmigrate/start -X POST -d storageEngine=wiredTiger -d clusterPerc=50

XML Response

.  <s:messages>                                                                                                                          
    <s:msg type="INFO">SHC KV Store migration has been successfully triggered</s:msg>                                                   
  </s:messages>


shcluster/captain/kvmigrate/status

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://localhost:8099/services/shcluster/captain/kvmigrate/status


Check the status of a KV store storage engine migration.

GET

Check the status.

Request parameters
None.

Returned values

Name Description
clusterPerc Percentage of cluster members that have completed migration.
migrationID ID number for the migration.
migrationStartTime Timestamp that the migration began.
peerRetryCount Number of times that the peer failed to migrate and retried.
status Status of the migration.
storageEngine Target storage engine.

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8099/services/shcluster/captain/kvmigrate/status

XML Response

...
        <s:key name="general">
          <s:dict>
            <s:key name="clusterPerc">50</s:key>
            <s:key name="migrationID">1596760435</s:key>
            <s:key name="migrationStartTime">Thu Aug  6 17:33:55 2020</s:key>
            <s:key name="peerRetryCount">1</s:key>
            <s:key name="status">kvstore_migration_started</s:key>
            <s:key name="storageEngine">wiredTiger</s:key>
          </s:dict>
        </s:key>
        <s:key name="peerStatus">
          <s:dict>
            <s:key name="https://fool02:8099">
              <s:dict>
                <s:key name="attempt_number">1</s:key>
                <s:key name="migrationStatus">migration_in_progress</s:key>
              </s:dict>
            </s:key>
            <s:key name="https://fool02:8103">
              <s:dict>
                <s:key name="attempt_number">1</s:key>
                <s:key name="migrationStatus">migration_succeeded</s:key>
              </s:dict>
            </s:key>


shcluster/captain/kvmigrate/stop

This endpoint is available in Splunk Enterprise versions 9.3.x and lower only. It is deprecated and removed in Splunk Enterprise 9.4 and higher.

https://<host>:<mPort>/services/shcluster/captain/kvmigrate/stop


Stop the migration of your KV store storage engine.


POST

Stop the migration.

Request parameters
None.

Example request and response


XML Request

curl -k -u admin:changeme -X POST https://localhost:8099/services/shcluster/captain/kvmigrate/stop

XML Response

...
  <s:messages>                                                                                                                          
    <s:msg type="INFO">KV Store migration stop initiated</s:msg>                                                                        
  </s:messages>


storage/collections/config

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config


Access and create collections.

GET

List all collections.

Request parameters
None.

Returned values

Name Description
disabled Boolean indicating collection state. By default, the value is false, indicating that the collection is enabled.
profilingEnabled Boolean indicating profiling status of slow-running operations. By default, this value is false, meaning that profiling is disabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is true.

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:28:27-07:00</updated> <generator build="229629" version="6.2"/>

<author>
  <name>Splunk</name>
</author>
<link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
<link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
... opensearch nodes elided ...
<s:messages/>
<entry>
  <title>kvstoredemo</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/kvstoredemo</id>
  <updated>2014-09-02T11:28:27-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/kvstoredemo/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ... </s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>
<entry>
  <title>test</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
  <updated>2014-09-02T11:28:27-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ...</s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>


POST

Create a collection.

Request parameters

Name Type Description
name String Required. Collection name
profilingEnabled Boolean A collections.conf file property that affects profilingThresholdMs. Defaults to false. Enable profiling of slow-running operations by setting profilingEnabled to true.
profilingThresholdMs Number Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is true. Defaults to 100. Set to 0 to log all slow-running operations.

Returned values

Name Description
disabled Boolean indicating collection state. By default, the value is false, indicating that the collection is enabled.
profilingEnabled Profiling status of slow-running operations, profilingThresholdMs. Defaults to false.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is true.

Example request and response


XML Request

curl -k -u admin:changeme -d name=test1 https://localhost:8089/servicesNS/nobody/search/storage/collections/config

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:25:32-07:00</updated> <generator build="229629" version="6.2"/>

<author>
  <name>Splunk</name>
</author>
<link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
<link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
... opensearch nodes elided ...
<s:messages/>
<entry>
  <title>test1</title>
  <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test1</id>
  <updated>2014-09-02T11:25:32-07:00</updated>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="alternate"/>
  <author>
    <name>admin</name>
  </author>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="list"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1/_reload" rel="_reload"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="edit"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1" rel="remove"/>
  <link href="/servicesNS/nobody/search/storage/collections/config/test1/disable" rel="disable"/>
  <content type="text/xml">
    <s:dict>
      <s:key name="disabled">0</s:key>
      <s:key name="eai:acl"> ... elided ... </s:key>
      <s:key name="eai:appName">search</s:key>
      <s:key name="eai:userName">nobody</s:key>
      <s:key name="profilingEnabled">false</s:key>
      <s:key name="profilingThresholdMs">100</s:key>
    </s:dict>
  </content>
</entry>


storage/collections/config/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/config/{collection}

Access, delete, or update a specific {collection}.


DELETE

Delete a specific {collection}.

Request parameters
None

Returned values
None

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test -X DELETE

XML Response

...
 <title>collections-conf</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id>
 <updated>2014-06-02T20:08:11-07:00</updated>
 <generator build="210538" version="20140530"/>
 <author>
   <name>Splunk</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/>
 ... opensearch nodes elided ...
 <s:messages/>



GET

Access a specific {collection}.

Request parameters
None


Returned values

Name Description
disabled Boolean indicating collection state. By default, the value is 0, meaning that the collection is enabled.
field.<fieldName> Field type. One of the following values.
  • array
  • number
  • bool
  • string
  • cidr
  • time
accelerated_fields.<field_name> Field acceleration name and JSON definition.
enforceTypes Boolean indicating if data types are enforced when inserting data into the collection. Defaults to false.
profilingEnabled Profiling status of slow-running operations, affecting profilingThresholdMs. By default, the value is false, meaning that profiling is disabled. If true, profiling is enabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is true.
replicate Boolean indicating whether the collection is replicated on indexers. Defaults to false, meaning that this collection is not replicated, and lookups that depend on the collection will not be available. However, if you run a lookup command with local=true, local lookups will still be available.

When true, this collection is replicated on indexers.
replication_dump_maximum_file_size Indicates the maximum file size (in KB) for each dump file when replication_dump_strategy=auto. Defaults to 10240KB.

If this value is larger than concerningReplicatedFileSize, which is set in the distsearch.conf file, the value of concerningReplicatedFileSize is used instead.

KV Store does not pre-calculate the size of the records that will be written to disk, so the size of the resulting files can be affected by the max_rows_in_memory_per_dump setting in the limits.conf file.
replication_dump_strategy One of the following two values.
  • auto: Default. Dumps are stored in multiple files when the size of the collection exceeds the value of replication_dump_maximum_file_size.
  • one_file: Dump files are stored in a single file.


Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/collections/config/test

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:42:54-07:00</updated> <generator build="229629" version="6.2"/>

<author>

 <name>Splunk</name>
</author> <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/> <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/> ... opensearch nodes elided ... <s:messages/> <entry>

 <title>test</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
 <updated>2014-09-02T11:42:54-07:00</updated>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
 <author>
   <name>admin</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
 <content type="text/xml">
   <s:dict>
     <s:key name="disabled">0</s:key>
     <s:key name="eai:acl"> ... elided ... </s:key>
     <s:key name="eai:appName">search</s:key>
     <s:key name="eai:attributes"> ... elided ... </s:key>
     <s:key name="eai:userName">nobody</s:key>
     <s:key name="profilingEnabled">false</s:key>
     <s:key name="profilingThresholdMs">100</s:key>
   </s:dict>
 </content>
</entry>

POST

Update a specific {collection}.

Request parameters

Name Type Description
field.<fieldName> String Field type. One of the following values.
  • array
  • number
  • bool
  • string
  • cidr
  • time
accelerated_fields.<field_name> String, JSON (see description) The name of a field acceleration (string) and its definition, in JSON key value format. For example, accelerated_fields.my_accel = {"id": 1}

Returned values

Name Description
disabled Collection state:
true = disabled.
false = [Default] enabled.
field.<fieldName> Field type. One of the following values.
  • array
  • number
  • bool
  • string
  • cidr
  • time
accelerated_fields.<field_name> The name of a field acceleration (string) and its definition, in JSON key value format. For example, accelerated_fields.my_accel = {"id": 1}
profilingEnabled Profiling status of slow-running operations, affecting profilingThresholdMs. By default, the value is false, meaning that profiling is disabled. If true, profiling is enabled.
profilingThresholdMs Threshold for logging slow-running operations, in milliseconds. Applies only if profilingEnabled is true.
replicate Boolean indicating whether the collection is replicated on indexers. Defaults to false, meaning that this collection is not replicated, and lookups that depend on the collection will not be available. However, if you run a lookup command with local=true, local lookups will still be available.

When true, this collection is replicated on indexers.
replication_dump_maximum_file_size Indicates the maximum file size (in KB) for each dump file when replication_dump_strategy=auto. Defaults to 10240KB.

If this value is larger than concerningReplicatedFileSize, which is set in the distsearch.conf file, the value of concerningReplicatedFileSize is used instead.

KV Store does not pre-calculate the size of the records that will be written to disk, so the size of the resulting files can be affected by the max_rows_in_memory_per_dump setting in the limits.conf file.
replication_dump_strategy One of the following two values.
  • auto: Default. Dumps are stored in multiple files when the size of the collection exceeds the value of replication_dump_maximum_file_size.
  • one_file: Dump files are stored in a single file.


Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test -d 'accelerated_fields.foo={"a": 1}' -d 'accelerated_fields.bar={"b": -1}' -d "field.a=number" -d "field.b=cidr"

XML Response

...
<title>collections-conf</title> <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config</id> <updated>2014-09-02T11:50:57-07:00</updated> <generator build="229629" version="6.2"/>

<author>

 <name>Splunk</name>
</author> <link href="/servicesNS/nobody/search/storage/collections/config/_new" rel="create"/> <link href="/servicesNS/nobody/search/storage/collections/config/_reload" rel="_reload"/> ... opensearch nodes elided ... <s:messages/> <entry>

 <title>test</title>
 <id>https://localhost:8089/servicesNS/nobody/search/storage/collections/config/test</id>
 <updated>2014-09-02T11:50:57-07:00</updated>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="alternate"/>
 <author>
   <name>admin</name>
 </author>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="list"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/_reload" rel="_reload"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="edit"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test" rel="remove"/>
 <link href="/servicesNS/nobody/search/storage/collections/config/test/disable" rel="disable"/>
 <content type="text/xml">
   <s:dict>
     <s:key name="disabled">0</s:key>
     <s:key name="eai:acl"> ... elided ... </s:key>
     <s:key name="eai:appName">search</s:key>
     <s:key name="eai:userName">nobody</s:key>
     <s:key name="accelerated_fields.foo">{"a": 1}
accelerated_fields.bar={"b": -1} field.a=number field.b=cidr</s:key>

     <s:key name="profilingEnabled">false</s:key>
     <s:key name="profilingThresholdMs">100</s:key>
   </s:dict>
 </content>
</entry>


storage/collections/data/{collection}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}


Access and manage items of a collection.


DELETE

Delete items in the {collection} or delete an entire collection.

Usage details
Use the query parameter to specify which entries to delete. If no query is provided, all entries in the collection are deleted.

Request parameters

Name Type Description
query JSON object Query JSON object.
Conditional operators: $gt, $gte, $lt, $lte, and $ne
Logical operators: $and, $or, and ,$not (invert conditional operators)
Examples:
query={"title":"Item"} (Select all documents with property title that has value Item)
query={"price":{"$gt":5}} (Select all documents with price greater than 5)

Returned values
None

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA -X DELETE

XML Response
No values returned.


GET

Access a specific {collection}.

Request parameters

Parameter order in the request does not matter. Parameters are always applied in the order of sort, skip, and limit.

Name Type Description
fields String Comma-separated list of fields to include (1) or exclude (0). A fields value cannot contain both include and exclude specifications except for exclusion of the _key field. Examples:
fields=firstname,surname (Include only firstname, surname, and _key fields)
fields=firstname,surname,_key:0 (Include only the firstname and surname fields)
fields=address:0 (Include all fields except the address field)
shared Boolean Defaults to false. Set to true to return records for the specified user as well as records for the nobody user.
limit Number Maximum number of items to return. For example, to return five items, use
limit=5.
skip Number Number of items to skip from the start. For example, to skip the first ten items, use
skip=10.
sort String Sort order. Examples:
sort=surname (Sort by surname, ascending)
sort=surname,firstname (Sort by surname, ascending, after firstname, ascending)
sort=surname:-1,firstname:1 (Sort by surname, descending, after firstname, ascending
sort=surname:1,first name (Sort by surname, ascending, after firstname, ascending
query JSON object Query JSON object.
Conditional operators: $gt, $gte, $lt, $lte, and $ne
Logical operators: $and, $or, and ,$not (invert conditional operators)
Examples:
query={"title":"Item"} (Select all documents with property title that has value Item)
query={"price":{"$gt":5}} (Select all documents with price greater than 5)

Returned values
The response includes a JSON document.

Example

[ { "myKey" : "abc", "_user" : "nobody", "_key" : "5410be5441ba15298e4624d1" } ]

POST

Insert an item into the {collection}.


Usage details
The Content-Type header must be application/json.

The _key is autogenerated, if not manually specified.

Request parameters
Make sure that the item to add is a JSON-formatted document, such as the following example. 

{ "name": "A" }

Returned values
The response includes a JSON-formatted document key, such as the following example. 

{
    "_key": "530bc62fc9a6577fdf13651f"
}

Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA -H "Content-Type: application/json" -d '{ "myKey": "abc",  "myOtherKey": "abcdef"}'

XML Response

{"_key":"5410be5441ba15298e4624d1"}


storage/collections/data/{collection}/{key}

https://<host>:<mPort>/servicesNS/{owner}/{app}/storage/collections/data/{collection}/{key}

Access and manage a specific {key} item in a {collection}.


DELETE

Delete a collection item.

Request parameters
None

Returned values
None


Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/collections/data/testCollectionA/5410caf041ba15298e4624d6 -X DELETE


XML Response
No values returned.

GET

Access a collection item.

Request parameters
None

Returned values
The response includes a JSON-formatted {key} document, such as the following example. 

{"myKey" : "abc", "myOtherKey" : "uvwxyz", "_user" : "nobody", "_key" : "5410c8dc41ba15298e4624d5"}

Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/5410c8dc41ba15298e4624d5

XML Response

{ "myKey" : "abc", "myOtherKey" : "uvwxyz", "_user" : "nobody", "_key" : "5410c8dc41ba15298e4624d5" }

POST

Update a collection item.

Usage details
The header Content-Type must be application/json.

Request parameters
Pass in a JSON-formatted document for the {key} that you are updating. For example, use { "myKey": "fizz"}.

Returned values
The response includes a JSON-formatted {key} such as the following example. 

{"_key":"5410c8dc41ba15298e4624d5"}


Example request and response


XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/5410c8dc41ba15298e4624d5 -H "Content-Type: application/json" -d '{ "myKey": "fizz"}'

XML Response

{"_key":"5410c8dc41ba15298e4624d5"}


storage/collections/data/{collection}/batch_find

https://<host>:<mPort>/servicesNS/storage/collections/data/{collection}/batch_find

Perform multiple queries in a batch.


POST

Perform multiple queries in a batch.

Request parameters
Pass in the JSON array of queries to run. Parameter order in the request does not matter. Parameters are always applied in the order of sort, skip, and limit.

Name Type Description
fields String Comma-separated list of fields to include (1) or exclude (0). A fields value cannot contain both include and exclude specifications except for exclusion of the _key field. Examples:
fields=firstname,surname (Include only firstname, surname, and _key fields)
fields=firstname,surname,_key:0 (Include only the firstname and surname fields.
fields=address:0 (Include all fields except the address field)
shared Boolean Defaults to false. Set to true to return records for the specified user as well as records for the nobody user.
limit Number Maximum number of items to return. For example, to return five items, use
limit=5.
skip Number Number of items to skip from the start. For example, to skip the first ten items, use
skip=10.
sort String Sort order. Examples:
sort=surname (Sort by surname, ascending)
sort=surname,firstname (Sort by surname, ascending, after firstname, ascending)
sort=surname:-1,firstname:1 (Sort by surname, descending, after firstname, ascending
sort=surname:1,first name (Sort by surname, ascending, after firstname, ascending
query JSON object Query JSON object.
Conditional operators: $gt, $gte, $lt, $lte, and $ne
Logical operators: $and, $or, and ,$not (invert conditional operators)
Examples:
query={"title":"Item"} (Select all documents with property title that has value Item)
query={"price":{"$gt":5}} (Select all documents with price greater than 5)

Returned values
The response includes a JSON-formatted array of an array of records. The first array of records is the result of the first query, the second array of records is the result of the second query, and so on.

Example request and response

XML Request

queries='['
queries+='{"query": {"myKey": "def"}},'
queries+='{"query": {"myKey": "abc"}},'
queries+='{"query": {"myKey": "jkl"}},'
queries+='{"shared": true, "fields": {"myKey": 1}, "sort": [{"myKey": 1}], "limit": 2}'
queries+=']'
curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/batch_find -H "Content-Type: application/json" -d "$queries"

XML Response

[
  [{"myKey":"def","_user":"nobody","_key":"f3jel0421dsw1024f3fd1a78"}],
  [{"myKey":"abc","_user":"nobody","_key":"b2jcw0405saw1221177aa78f"}],
  [{"myKey":"jkl","_user":"nobody","_key":"3b4b0d9ef63cc83959e5ed77"}],
  [{"myKey":"abc"},{"myKey":"def"}]
]


storage/collections/data/{collection}/batch_save

https://<host>:<mPort>/servicesNS/storage/collections/data/{collection}/batch_save

Perform multiple save operations in a batch.


POST

Perform multiple save operations in a batch.

Usage details

The request header Content-Type must be application/json.

If a document does not have a _key field or store does not have a matching document ID, a _key new document is created.

If an operation fails, the server stops batch processing and returns an HTTP status code other than 200.


Request parameters
Pass in the JSON array of documents to save.

Returned values
The response includes a JSON-formatted list of keys added, such as the following example. 

[ "5410c43241ba15298e4624d3", "5418c54e41ba152267763c01" ]


Example request and response

XML Request

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/search/storage/collections/data/testCollectionA/batch_save -H "Content-Type: application/json" -d '[{ "_key": "5410c43241ba15298e4624d3", "name": "AAAAAAAA" },{ "name": "A" }]'

XML Response

[ "5410c43241ba15298e4624d3", "5418c54e41ba152267763c01" ]


Last modified on 07 November, 2024
Knowledge endpoint descriptions   License endpoint descriptions

This documentation applies to the following versions of Splunk® Enterprise: 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.1.7, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.2.4, 9.3.0, 9.3.1, 9.3.2

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters