Splunk® Enterprise

Managing Indexers and Clusters of Indexers

Splunk Enterprise version 7.3 is no longer supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Configure the peer indexes in an indexer cluster

You configure indexes by editing the indexes.conf file. This file determines an indexer's set of indexes, as well as the size and attributes of its buckets. Since all peers in a cluster must use the same set of indexes (except for limited purposes, described later), the indexes.conf file should ordinarily be the same across all peers.

The cluster peers deploy with a peer-specific default indexes.conf file that handles basic cluster needs. If you want to add indexes or change bucket behavior, you edit a new indexes.conf file in a special location on the master and then distribute the file simultaneously to all the peers.

Important: You cannot use Splunk Web or the CLI to configure index settings on peer nodes. You must edit indexes.conf directly.

All peers must use the same set of indexes.conf files

The set of indexes.conf files should ordinarily be identical across all peers in a cluster. In particular, all peers must use the same set of clustered indexes. This is essential for index replication to work properly. (The master node, on the other hand, has its own, separate indexes.conf file, because it indexes only its own internal data.) There is a limited exception to this restriction, which is described a bit later.

When you first create the cluster, the master distributes a special default indexes.conf file to each of the peers. This version supplements the standard default indexes.conf that all indexers get. The peer-specific default indexes.conf turns on replication for the main index, as well as the internal indexes, such as _audit and _internal.

Depending on your system, you might also need to edit and distribute a modified indexes.conf to the peers, to accommodate additional indexes or changes to bucket attributes. To ensure that all peers use the same indexes.conf, you must use the master node to distribute the file to all the peers as a single process.This process, known as the configuration bundle method, is described in Update common peer configurations and apps.

You must also use the configuration bundle method to distribute apps across all the peers. These apps might contain their own indexes.conf files, which will layer appropriately with any non-app version of the file that you might also distribute to the peers. For information on app distribution, read Manage app deployment across all peers.

Note: Under limited circumstances (for example, to perform local testing or monitoring), you can create an indexes.conf for a single peer only. Such an index will not get replicated. The single-peer indexes.conf supplements, but does not replace, the common version of the file that all peers get. See Add an index to a single peer for details.

Configure a set of indexes for the peers

There are two steps to configuring indexes across the set of peers:

1. Edit a common indexes.conf file on the master.

2. Use the master to distribute the file across the set of peers.

These two steps are described below.

1. Edit indexes.conf

For details on configuring indexes.conf, read the topics in the chapters Manage indexes and Manage index storage in this manual. For a list of all indexes.conf attributes, see the indexes.conf specification file in the Admin Manual.

For the most part, you edit the cluster peer indexes.conf in the same way as for any indexer. However, there are a few differences to be aware of.

The indexes.conf repFactor attribute

When you add a new index stanza, you must set the repFactor attribute to "auto". This causes the index's data to be replicated to other peers in the cluster. For example:

[idx1]
repFactor = auto
homePath = $SPLUNK_DB/$_index_name/db
coldPath = $SPLUNK_DB/$_index_name/colddb
thawedPath = <path for thawed buckets>
...

Note: By default, repFactor is set to 0, which means that the index will not be replicated. For clustered indexes, you must set it to "auto".

The only valid values for repFactor are 0 and "auto".

Resetting repFactor from "auto" to 0 will stop further replication, but it will not automatically remove copies of already replicated buckets. In addition, searches across buckets with multiple copies will return duplicate events. To free up associated disk space and eliminate the possibility of duplicate events, you must remove the excess copies manually.

Specify the index path attributes with forward-slash directory separators

In heterogeneous environments, it is possible that the master node's operating system could use a different convention for specifying directory paths from the peer nodes' operating system. This presents a problem because you edit the indexes.conf file on the master but then you distribute it to the peers.

For example, if you have a Windows master and a set of Linux peers, the normal way to specify the homePath on the Windows master, where the file gets edited, would be to use the Windows backward-slash convention as a directory separator, while the Linux peers, where the file gets distributed, require forward slashes.

To deal with this possibility, the best practice is to always use forward slashes when specifying directory paths in in the index path attributes, no matter which operating systems your master and peers use. For example:

homePath = $SPLUNK_DB/$_index_name/db

Splunk Enterprise always accepts the forward slash as a directory separator.

2. Distribute the new indexes.conf file to the peers

After you edit indexes.conf, you need to distribute it to the cluster's set of peer nodes. To learn how to distribute configuration files, including indexes.conf, across all the peers, read Update common peer configurations and apps.

For information about other types of peer configuration, including app distribution, read Peer node configuration overview.

View the indexes

To see the set of indexes on your peer nodes, click the Indexes tab on the master dashboard. See View the master dashboard.

Note: A new index appears under the tab only after it contains some data. In other words, if you configure a new index on the peer nodes, a row for that index appears only after you send data to that index.

Last modified on 29 September, 2020
Manage app deployment across all peers   Update common peer configurations and apps

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters