Manage indexes in Splunk Cloud Platform

The Admin Config Service (ACS) API supports programmatic self-service management of indexes in Splunk Cloud Platform. You can use the ACS API to create new indexes, update index configurations, and delete indexes directly on your Splunk Cloud Platform deployment without assistance from Splunk Support.

The Terraform Provider for Splunk Cloud Platform now supports ACS index operations. To access the Splunk Cloud Platform Terraform Provider (terraform-provider-scp), see the Terraform Registry.

Requirements

To manage indexes using the ACS API:

• You must have Splunk Cloud Platform version 8.2.2106 or higher.
• Your role must have the capabilities required to access the ACS API endpoint. The sc_admin role has all required capabilities by default. For a list of required capabilities, see Manage ACS API access with capabilities.
• Your deployment must have one or more separate search heads or a search head cluster. ACS is not supported on single instance deployments.

Set up the ACS API

Before using the ACS API, you must download the ACS Open API 3.0 specification, which includes the parameters, response codes, and other data you need to work with the ACS API. You must also create an authentication token in Splunk Cloud Platform for use with ACS endpoint requests. For details on how to set up the ACS API for index management, see Set up the ACS API.

Manage indexes using the ACS API

The ACS API does not support management of Splunk internal indexes. For help managing internal indexes, contact Splunk Support.

View existing indexes

To view a list of existing indexes, send an HTTP GET request to the following endpoint:

admin.splunk.com/{stack_name}/adminconfig/v2/indexes

For example:

curl 'https://admin.splunk.com/{stack}/adminconfig/v2/indexes' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' 

The request returns a list of indexes, including the configuration parameter values for each index. For example:

[
   {
       "name": "history",
       "datatype": "event",
       "searchableDays": 7,
       "maxDataSizeMB": 0,
       "totalEventCount": "0",
       "totalRawSizeMB": "0"
   },
   {
       "name": "lastchanceindex",
       "datatype": "event",
       "searchableDays": 1095,
       "maxDataSizeMB": 0,
       "totalEventCount": "0",
       "totalRawSizeMB": "0"
   },
{
       "name": "splunklogger",
       "datatype": "event",
       "searchableDays": 1095,
       "maxDataSizeMB": 0,
       "totalEventCount": "0",
       "totalRawSizeMB": "0"
   }
]

By default, the list indexes request returns a maximum count of 30 indexes. To change the count, you can specify a count value up to a maximum of 100. A count value of 0 lists all indexes.

If you have more than 100 indexes, you can specify an offset value to list additional indexes. For example, to list indexes 100-200, specify an offset value of 100. For example:

curl 'https://admin.splunk.com/{stack}/adminconfig/v2/indexes?offset=100&count=100'

For endpoint details, see indexes in the ACS endpoint reference.

Create a new index

To create a new index, send an HTTP POST request to the indexes endpoint, specifying the index name and other configuration parameters in the request payload. Configurable parameters include name, datatype (metric or event), maxDataSizeMB, searchableDays, SplunkArchivalRetentionDays, and selfStorageBucketPath.

If you only specify the index "name" without specifying any other index configuration parameters, ACS automatically populates the configuration parameters with the following pre-determined values "datatype": "event", "maxDataSizeMB": 0" (unlimited), and "searchableDays": 90".

For example:

curl -X POST 'https://admin.splunk.com/{stack}/adminconfig/v2/indexes'
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
   "name": "testindex"
}'

The request returns index configuration details in the response body, with a 202 status code, which indicates the request was accepted, but index creation is still in progress, due to the asynchronous index creation process. For example:

{
  "datatype": "event",
  "maxDataSizeMB": 0,
  "name": "testindex",
  "searchableDays": 90
}

Create a new index with DDAA or DDSS

When you create an index using the ACS API, you can optionally enable one of two dynamic data storage options available in Splunk Cloud Platform:

• Dynamic Data Active Archive (DDAA). See Store expired Splunk Cloud Platform data in a Splunk-managed archive.
• Dynamic Data Self Storage (DDSS): See Store expired Splunk Cloud Platform data in your private archive.

Create an index with DDAA

To create an index with DDAA enabled, you must specify the splunkArchivalRetentionDays value in the request body. When specifying the splunkArchivalRetentionDays value, note the following:

• The value of splunkArchivalRetentionDays must be greater than the SearchableDays value, as the archival retention is the total retention period including the searchable retention.
• The splunkArchivalRetentionDays value must be less than or equal to the total to the maxDataArchiveRetentionPeriod for the deployment, which has a maximum value of 10 years (3650 days).
• The splunkArchivalRetentionDays is not a rolling period. Calculate the retention period from the index creation date. This also applies to update (PATCH) requests.

For example:

curl --location --request POST 'https://admin.splunk.com/<stack>/adminconfig/v2/indexes' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <JWT Token>' \
--data-raw '{
"name": "test-ddaa",
"Datatype": "event",
"MaxDataSizeMB": 512,
"SearchableDays": 10,
"SplunkArchivalRetentionDays": 100
}'

The response includes the specified splunkArchivalRetentionDays value. For example:

Response:
{
"datatype": "event",
"maxDataSizeMB": 512,
"name": "test-ddaa",
"searchableDays": 10,
"splunkArchivalRetentionDays": 100
}

Create an index with DDSS

To create an index with DDSS enabled, you must specify the selfStorageBucketPath value in the following format: "s3://selfStorageBucket/selfStorageBucketFolder", where SelfStorageBucketFolder is optional, as you can store data buckets at root.

Before you can create an index with DDSS enabled, you must configure a self-storage location for your deployment. For instructions on how to configure self-storage locations, see Store expired Splunk Cloud Platform data in your private archive.

For example:

curl --location --request POST 'https://admin.splunk.com/<stack:myteststack>/adminconfig/v2/indexes' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <JWT token> \
--data-raw '{ 
"name": "test-ddss",
"Datatype": "event",
"MaxDataSizeMB": 512,
"SearchableDays": 100,
"selfStorageBucketPath":"s3://myteststack-wye6m83vqiuf-testnew/test" }'

The response includes the selfStorageBucketPath value. For example:

{
"datatype": "event",
"maxDataSizeMB": 512,
"name": "test-ddss",
"searchableDays": 100,
"selfStorageBucketPath": "s3://myteststack-wye6m83vqiuf-testnew/test"
}

For self-storage locations on AWS, the selfStorageBucketPath URI begins with the prefix "s3", as shown in the preceding example. For self-storage locations on GCP, the selfStorageBucketPath URI begins with the prefix "gs". For example: "gs://myteststack-wye6m83vqiuf-testnew/test".

For endpoint details, see indexes in the ACS endpoint reference.

Confirm index creation

To confirm that your new index was created successfully, send an HTTP GET request to the index/{name} endpoint, specifying the index name. For example:

curl 'https://admin.splunk.com/{stack}/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' 

If the request returns "code": "404-index-not-found", index creation is still in progress. For example:

{
   "code": "404-index-not-found",
   "message": "testindex index not found"
}

It might take a few minutes for the index creation process to complete. Repeat the GET request until the status code is 200.

When index creation is complete, the request returns the configuration details in the response body, with a 200 status code. For example:

{
    "datatype": "event",
    "maxDataSizeMB": 0,
    "name": "testindex",
    "searchableDays": 90,
    "totalEventCount": "0",
    "totalRawSizeMB": "0"
 }

View an individual index

To view configuration details of a specific index, send an HTTP GET request to the indexes/{name} endpoint, specifying the name of the index. For example:

curl 'https://admin.splunk.com/{stack}/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' 

The request returns the index configuration details for the specific index. For example:

{
   "name": "testindex",
   "datatype": "event",
   "searchableDays": 90,
   "maxDataSizeMB": 1024,
   "totalEventCount": "0",
   "totalRawSizeMB": "0"
}

For endpoint details, see indexes/{name} in the ACS endpoint reference.

Update an index

To update the index configuration, send a PATCH request to the indexes/{name} endpoint, specifying the name of the index, and the configuration values that you want to update. The PATCH request supports updates to the searchableDays, maxDataSizeMB, splunkArchivalRetentionDays, and selfStorageBucketPath parameters. For example, to update searchableDays to 50 and maxDataSizeMB to 2048, send a request specifying the updated values as follows:

curl -X PATCH 'https://admin.splunk.com/lighthearted-lemur-23e/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
   "searchableDays": 50,
   "maxDataSizeMB": 2048
}'

ACS does not support updates to the index name and datatype parameters.

The request returns the index configuration values, with a 202 status code, which indicates the request has been accepted, but the index update is still in progress due to the asynchronous index update process.

To confirm the index update is complete, send an HTTP GET request to the indexes/{name} endpoint. Repeat the GET request until the response body shows the updated configuration values. For example:

{
   "name": "testindex",
   "datatype": "event",
   "searchableDays": 50,
   "maxDataSizeMB": 2048,
   "totalEventCount": "0",
   "totalRawSizeMB": "0"
}

Update an index with DDAA or DDSS

You can send an update (PATCH) request to enable or change DDAA or DDSS settings for an index. ACS supports the following PATCH operations for DDAA/DDSS:

• Update an index to enable DDAA or DDSS.
• Update DDAA settings (change the archival retention period) for an index already enabled with DDAA.
• Update DDSS settings (change the self-storage location) for an index already enabled with DDSS.

ACS supports bulk updates of DDAA and DDSS settings using the ACS CLI. For more information, see Update indexes in bulk.

Update an index with DDAA

Send an HTTP PATCH request to the indexes/{name} endpoint, specifying a non-zero value for the splunkArchivalRetentionDays parameter in the request body. Use this operation to enable DDAA for an index, or to increase/decrease the splunkArchivalRetentionDays value for an index already enabled with DDAA. For example:

curl -X PATCH 'https://admin.splunk.com/lighthearted-lemur-23e/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
   "SplunkArchivalRetentionDays": 100
}'

For more information on the splunkArchivalRetentionDays setting, see Create a new index with DDAA or DDSS.

Update an index with DDSS

Send an HTTP PATCH request to the indexes/{name} endpoint, specifying the selfStorageBucketPath parameter in the request body, in the format "s3://selfStorageBucket/selfStorageBucketFolder", where selfStorageBucketFolder is optional, as you can store data buckets at root. Use this operation to enable DDSS for an index, or to change the self-storage location for an index already configured with DDSS. For example:

curl -X PATCH 'https://admin.splunk.com/lighthearted-lemur-23e/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
   "selfStorageBucketPath": "s3://stack-prefix-bucket-name/folder-name"
}'

You cannot update an index to disable DDAA or DDSS, nor can you update an index to change from DDSS to DDAA, or vice versa. To disable DDAA/DDSS or change the dynamic data storage type for an index, you must use the Splunk web UI.

For endpoint details, see indexes/{name} in the ACS endpoint reference.

Delete an index

To delete an index, send an HTTP GET request to the indexes/{name} endpoint, specifying the name of the index you want to delete. For example:

curl -X DELETE 'https://admin.splunk.com/lighthearted-lemur-23e/adminconfig/v2/indexes/testindex' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' 

To confirm that the index has been deleted, send a GET request to the indexes endpoint and check that the index is no longer present in the list of indexes. Or send a GET request to the indexes/{name} endpoint, which returns a message stating that the index is not found.

Index deletion can in some cases trigger a rolling restart of your Splunk Cloud Platform deployment.

For endpoint details, see indexes/{name} in the ACS endpoint reference.