Resync the KV store

When a KV store member fails to transform its data with all of the write operations, then the KV store member might be stale. To resolve this issue, you must resynchronize the member.

Identify a stale KV store member

Check the REST introspection /serverstatus endpoint to identify a stale KV store member. You can use cURL to make a GET request for the endpoint.

curl -k -u user:pass https://<host>:<mPort>/services/server/introspection/kvstore/serverstatus

If the endpoint fails to return introspection data, then the member is stale.

For more information about the REST API, see Basic Concepts in the REST API User Manual.

For more information about introspection endpoints, see Introspection endpoint descriptions in the REST API Reference Manual.

Resync stale KV store members

If more than half of the members are stale, then recreate the cluster. See Back up KV store.

If fewer than half of the members are stale, resync each KV store member individually.

1. Stop the search head that has the stale KV store member.
2. Run the command splunk clean kvstore --local.
3. Restart the search head.