Configure the search head cluster
This topic describes how to configure the behavior of the search head cluster itself. It does not describe how to configure the search-time environment of the cluster members, such as the set of saved searches, dashboards, and apps that the members have access to. For information on configuring the search-time environment, see the chapter "Update search head cluster members".
The members store their cluster configurations in their local
server.conf files, located under
$SPLUNK_HOME/etc/system/local/. See the server.conf specification file for details on all available configuration attributes.
Remember these key points while reading this topic:
- The essential configuration occurs when you initialize each member during the deployment process.
- Search head clustering has a large number of configuration settings available. With a few exceptions, you should not change these settings from their initial or default values without guidance from Splunk Support.
- You must maintain identical settings across all members, except as noted.
- When you do change a setting across all members, you must restart all the members at approximately the same time.
You can set all essential configurations during the deployment process, when you initialize each member. These are the key configuration attributes that you can or must set for each cluster member during initialization:
- The member's URI. See "Deploy a search head cluster".
- The member's replication port. See "Deploy a search head cluster".
- The cluster's replication factor. See "Choose the replication factor for the search head cluster".
- The cluster's security key. See "Set a security key for the search head cluster".
- The deployer location. See "Point the cluster members to the deployer".
- The cluster's label. See "Deploy a search head cluster".
Caution: It is strongly recommended that you set all these attributes during initialization and do not later change them. See "Deploy a search head cluster".
Post-initialization configuration changes
The main configuration changes that you can safely perform on your own, post-initialization, are the ad hoc search settings. There are two of these: one for specifying whether a particular member should run ad hoc searches only, and another for specifying whether the member currently functioning as the captain should run ad hoc searches only. The captain will not assign scheduled searches to ad hoc members. See "Configure a cluster member to run ad hoc searches only".
You can also temporarily switch to a static captain, as a work around for disaster recovery. See "Use static captain to recover from loss of majority."
Caution: Do not edit the
id attribute in the
[shclustering] stanza. The system sets it automatically. This attribute must conform to the requirements for a valid GUID.
Set the search head cluster label
You usually set the cluster label with the
splunk init command when you deploy the cluster. If you did not set it during deployment, you can later set it for the cluster by running this command on any one member:
splunk edit shcluster-config -shcluster_label <label>
You do not need to restart the member after setting the label.
Note: If you set the label on a cluster member, you must also set it on the deployer. See "Configure the deployer."
-shcluster_label parameter is useful for identifying the cluster in the monitoring console. See "Set cluster labels" in Monitoring Splunk Enterprise.
Maintain the same configuration settings across all members
server.conf attributes for search head clustering must have the same values across all members, with these exceptions:
If any configuration values other than these ones vary from member to member, then the behavior of the cluster will change depending on which member is currently serving as captain. You do not want that to occur.
Most of the configuration occurs during initial cluster deployment, through the CLI
splunk init command. To perform further configuration later, you have two choices:
- Use the CLI
splunk edit shcluster-configcommand.
- Edit the
It is generally simpler to use the CLI.
Caution: You must make the same configuration changes on all members and then restart them all at approximately the same time. Because of the importance of maintaining identical settings across all members, do not use the
splunk rolling-restart command to restart, except when changing the
captain_is_adhoc_searchhead attribute, as described in "Configure a cluster member to run ad hoc searches only". Instead, run the
splunk restart command on each member.
Configure search head clustering with the CLI
You can use the CLI
splunk edit shcluster-config command to make edits to the
[shclustering] stanza in
server.conf. Specify each attribute and its configured value as a key value pair.
For example, to edit the
splunk edit shcluster-config -adhoc_searchhead true -auth <username>:<password>
The CLI confirms that the operation was successful and instructs you to restart
Note the following:
- You can use this command to edit any attribute in the
[shclustering]stanza except the
disabledattribute, which turns search head clustering on and off.
- You can only use this command on a member that has already been initialized. For initial configuration, use
splunk init shcluster-config.
Configure search head clustering by editing server.conf
You can also change attributes by directly editing
server.conf. The search head clustering attributes are located in the
[shclustering] stanza, with one exception: To modify the replication port, use the
Perform a rolling upgrade of a search head cluster
Choose the replication factor for the search head cluster
This documentation applies to the following versions of Splunk® Enterprise: 6.5.7, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 9.0.0