Splunk® Enterprise

Managing Indexers and Clusters of Indexers

Splunk Enterprise version 7.2 is no longer supported as of April 30, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Use indexer clusters to scale indexing

The main purpose of indexer clusters is to enable index replication. However, clusters can also be generally useful in scale-out deployment topologies as a way to manage multiple indexers, even when index replication isn't a requirement.

For example, say you want to create a deployment of three indexers and one search head, so that you can index larger quantities of data than a single indexer is capable of. The customary way of doing this, and the only way possible prior to Splunk Enterprise 5.0, is to set up each of the indexers independently, add in a search head, and then use a tool like deployment server to coordinate the indexer configurations.

With clustering, you can instead configure this deployment scenario as a cluster, with three peer nodes replacing the three independent indexers. Even if you don't need index replication and its key advantages like data availability and disaster tolerance, there are several reasons why it might be beneficial to use a cluster to coordinate multiple indexer instances:

  • Simplified management and coordination of indexer configuration (in place of using deployment server or performing manual updates). See "Update common peer configurations" for details.
  • Simplified set up and control of distributed search. See "Enable the search head".
  • Better insight into the state of your indexers through the clustering dashboards. See "View the master dashboard".
  • Ability to take advantage of additional cluster management capabilities as they're developed.

The main downsides of employing a cluster for scaling indexing capacity are these:

  • You must install an additional Splunk Enterprise instance to function as the cluster master node.
  • The cluster does not support heterogeneous indexers. All cluster nodes must be at the same version level. In addition, all peer nodes in a cluster must use the same indexes.conf configuration. For further details, see the next section, "Cluster peer management compared to deployment server".
  • You cannot use the deployment server to distribute configurations or apps across the cluster peers. For further details, see the next section, "Cluster peer management compared to deployment server".

Cluster peer management compared to deployment server

One useful cluster feature is the ability to manage and update the configuration for all indexers (peer nodes) from a central location, the master node. In that respect, it's similar in function to the deployment server. Unlike the deployment server, however, cluster peer management does not have any concept of server classes. Because of this, and because of the way clusters coordinate their activities, you cannot specify different app or indexes.conf configurations for different groups of indexers. (All peer nodes in a cluster must use the same indexes.conf configurations, as well as some other configurations, as described in "Peer node configuration overview".) If you need to maintain a heterogeneous set of indexers, you cannot employ clusters for scaling purposes.

On the other hand, the configuration bundle method used to download updates to peer nodes has certain advantages over the deployment server. Specifically, it not only distributes updates, it also validates them on the peers, and then (when necessary) initiates a rolling restart of the peers. See "Update common peer configurations" for details.

Important: Do not use deployment server or third-party distributed configuration management software, such as Puppet or Chef, to deploy updates directly to peer nodes. You can use such tools to deploy updates to the master, which then deploys those updates to the peers. See "Use deployment server to distribute the apps to the master".

Configure a cluster for scale-out deployment

To set up a cluster for scale-out deployment, without index replication, just set both the replication factor and search factor to 1. This causes the cluster to function purely as a coordinated set of Splunk Enterprise instances, without data replication. The cluster will not make any duplicate copies of the data, so you can keep storage size and processing overhead to a minimum.

Last modified on 22 September, 2020
Prepare the peers for index replication   Migrate non-clustered indexers to a clustered environment

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters