How indexer cluster nodes start up
This topic describes what happens when:
- the manager node starts
- a peer node joins a new cluster
- a peer node joins an existing cluster
When the manager node starts
When a manager node comes online (either the first time or subsequently), it begins listening for cluster peers. Each online peer registers with the manager, and the manager adds it to the cluster. The manager waits until the replication factor number of peers register, and then it starts performing its functions.
When you first deploy the cluster, you must enable the manager before enabling the peer nodes, as described in "Indexer cluster deployment overview". The manager blocks indexing on the peers until you enable and restart the full replication factor number of peers.
If you subsequently restart the manager, it waits for a quiet period to allow all peers the opportunity to register with it. Once the quiet period ends and the replication factor number of peers register with it, the manager can start performing its coordinating functions, such as rebalancing the primary bucket copies and telling peers where to stream copies of incoming data. Therefore, you must make sure that there are at least replication factor number of peers running when you restart the manager.
After the quiet period is over, you can view the manager node dashboard for information on the status of the cluster.
For more information on what occurs when a manager goes down and then restarts, see "What happens when the manager node goes down".
When a peer joins a new cluster
When you initially deploy a cluster, you must first enable the manager and then enable the peer nodes, as described in "Indexer cluster deployment overview". The manager blocks indexing on the peers until you enable and restart the full replication factor number of peers.
Each peer registers with the manager when it comes online, and the manager automatically distributes the latest configuration bundle to it. The peer then validates the configuration bundle locally. The peer will only join the cluster if bundle validation succeeds.
The peer starts indexing data after the replication factor number of peers join the cluster.
When a peer joins an existing cluster
A peer can also come online at some later time, when the cluster is already up and running with a manager and the replication factor number of peers. The peer registers with the manager when it comes online, and the manager distributes the latest configuration bundle to it. The peer validates the configuration bundle locally. The peer joins the cluster only if bundle validation succeeds.
Note: Adding a new peer to an existing cluster causes a rebalancing of primary bucket copies across the set of existing peer nodes, as described in "Rebalance the indexer cluster primary buckets". However, the new peer won't get any of those primary copies, because the manager performs rebalancing only across the set of searchable copies, and a new peer has no searchable copies when it starts up. The peer can participate in future bucket replication, but the manager does not automatically shift bucket copies or reassign primaries from existing peers to the new peer.
How indexer clusters handle report and data model acceleration summaries
What happens when a peer node goes down
This documentation applies to the following versions of Splunk® Enterprise: 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.1.0, 9.1.1