Splunk® Enterprise

Managing Indexers and Clusters of Indexers

Acrobat logo Download manual as PDF


Splunk Enterprise version 7.2 is no longer supported as of April 30, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Remove excess bucket copies from the indexer cluster

Excess bucket copies are copies that exceed the cluster's replication factor or search factor. For example, if the cluster has a replication factor of 3, each bucket should optimally have exactly three copies residing across the set of peer nodes. If one bucket has four copies, that bucket has one excess copy.

Excess copies do not interfere with the operation of the cluster, but they are unnecessary and require extra disk space.

You can view and remove excess bucket copies from the Master dashboard or from the CLI.

How excess copies originate

Excess copies can result from peers leaving the cluster and then returning to it. When a peer goes down, the cluster initiates bucket fixing activities to compensate for any copies on that peer, because those copies are no longer available to the cluster. The goal of bucket fixing is return the cluster to the complete state, where each bucket has a replication factor number of copies and a search factor number of searchable copies.

If the peer later returns to the indexer cluster, any bucket copies that the peer retained while down are once again available to the cluster. This can result in the cluster maintaining excess copies of some buckets, as described in the topic What happens when a peer node comes back up.

In effect, a returning peer can cause the cluster to store more copies of some buckets than are needed to fulfill the replication factor and, possibly, the search factor as well. It can sometimes be useful to keep the extra copies around, as that topic explains, but you can save disk space by instead removing them.

Use the Master dashboard

To view or remove excess bucket copies:

1. On the master node, click Settings on the upper right side of Splunk Web.

2. In the Distributed Environment group, click Indexer clustering.

This takes you to the Master dashboard.

3. Select the Indexes tab.

4. Click the Bucket Status button.

This takes you to the Bucket Status dashboard.

5. Select the Indexes with Excess Buckets tab.

This tab provides a list of indexes with excess bucket copies. It enumerates both buckets with excess copies and buckets with excess searchable copies. It also enumerates the total excess copies in each category. For example, if your index "new" has one bucket with three excess copies, one of which is searchable, and a second bucket with one excess copy, which is non-searchable, the row for "new" will report:

  • 2 buckets with excess copies
  • 1 bucket with excess searchable copies
  • 4 total excess copies
  • 1 total excess searchable copies

If you want to remove the excess copies for a single index, click the Remove button on the right side of the row for that index.

If you want to remove the excess copies for all indexes, click the Remove All Excess Buckets button.

Use the CLI

The Splunk CLI has two commands that help manage and remove excess bucket copies. You can run these commands either across the entire set of indexes or on just a single index.

Determine whether the cluster has extra copies

To find out how many buckets have extra copies, including extra searchable copies, run this command from the master:

splunk list excess-buckets [index-name]

The output from splunk list excess-buckets looks like this:

index=_audit
       Total number of buckets=4
       Number of buckets with excess replication copies=0
       Number of buckets with excess searchable copies=0
       Total number of excess replication copies across all buckets=0
       Total number of excess searchable copies across all buckets=0
index=_internal
       Total number of buckets=4
       Number of buckets with excess replication copies=0
       Number of buckets with excess searchable copies=0
       Total number of excess replication copies across all buckets=0
       Total number of excess searchable copies across all buckets=0
index=main
       Total number of buckets=5
       Number of buckets with excess replication copies=5
       Number of buckets with excess searchable copies=5
       Total number of excess replication copies across all buckets=10
       Total number of excess searchable copies across all buckets=5

Remove extra bucket copies

To remove all extra bucket copies from the cluster (or from one index on the cluster), run this command from the master:

splunk remove excess-buckets [index-name]

The master determines which peers to remove the extra copies from. This is not configurable, and the extras will not necessarily be removed from the peer that has most recently returned to the cluster.

Last modified on 13 February, 2019
PREVIOUS
Rebalance the indexer cluster
  NEXT
Put a peer into detention

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters