Configure multisite indexer clusters with the CLI

Read this first

Before reading this topic, see:

• "Multisite indexer cluster deployment overview". This topic provides important background information about configuring a multisite cluster.

• "Configure the indexer cluster with the CLI". This topic explains the basics of using the CLI to configure a cluster. It focuses on single-site clusters, but most of its information is relevant to multisite clusters as well.

• "Configure multisite indexer clusters with server.conf". This topic provides useful information on configuring a multisite cluster, including details on the attributes corresponding to the command-line options described in the current topic.

Configure multisite cluster nodes

You configure instances as multisite cluster nodes with the splunk edit cluster-config command. After enabling an instance, you must restart it.

Site values

Site values identify the site on which a node resides. You assign a site value to each node in a multisite cluster.

Site values have the syntax:

>site<n>

where <n> is an integer in the range of 1 to 63: site1, site2, site3, ....

Note: In the case of a search head only, you can also set the site value to "site0". This setting disables search affinity for the search head.

Configure the manager node

Here is an example of a multisite configuration for a manager mode:

splunk edit cluster-config -mode manager -multisite true -available_sites site1,site2 -site site1 -site_replication_factor origin:2,total:3 -site_search_factor origin:1,total:2 -secret your_key

splunk restart

This example specifies that:

• the instance is a cluster manager node.
• the cluster is multisite.
• the cluster consists of two sites: site1 and site2.
• the manager is located on site1.
• the cluster's replication factor is the default "origin:2,total:3".
• the cluster's search factor is "origin:1,total:2".
• the manager, along with the other nodes in the cluster, uses "your_key" as its security key. The -secret flag modifies the pass4SymmKey setting in the [clustering] stanza of server.conf.

Note the following:

• Each cluster has only one manager.
• You must set multisite to true for multisite cluster managers.
• You must list all cluster sites with the available_sites attribute.
• You must set a site_replication_factor and a site_search_factor. For details, see "Configure the site replication factor" and "Configure the site search factor".

You might also need to adjust the single-site replication and search factors. See "How multisite configuration differs from single-site configuration." When the manager starts up for the first time, it blocks indexing on the peers until you enable and restart the full replication factor number of peers. For example, given a three-site cluster with "site_replication_factor = origin:2, site1:1, site2:2, site3:3, total:8", the manager blocks indexing until there are at least eight peers in total across all sites, including at least one in site1, two in site2, and three in site3.

Do not restart the manager while it is waiting for the peers to join the cluster. If you do, you will need to restart the peers a second time.

You do not need to restart the manager if you later change its site value.

Configure the peer nodes

To configure a peer node in a multisite cluster, you set a site attribute. All other configuration settings are identical to a peer in a single-site cluster.

Here is an example configuration for a multisite peer node:

splunk edit cluster-config -mode peer -site site1 -manager_uri https://10.160.31.200:8089 -replication_port 9887 -secret your_key

splunk restart

This example specifies that:

• the instance is a cluster peer node.
• the instance is located in site1. A peer can belong to only a single site.
• the peer's cluster manager node is located at 10.160.31.200:8089.
• the peer will use port 9887 to listen for replicated data streamed from the other peers. You can specify any available, unused port as the replication port. Do not re-use the management or receiving ports.

You do not need to restart the peer if you later change its site value.

Configure the search heads

To configure a search head for a multisite cluster, set the site parameter. All other settings are the same as for a search head in a single-site cluster.

You use different commands to configure a search head initially and to change its configuration later.

To initially configure a search head:

Use the splunk edit cluster-config command. Here is an example configuration for a multisite search head:

splunk edit cluster-config -mode searchhead -site site1 -manager_uri https://10.160.31.200:8089 -secret your_key

splunk restart

This example specifies that:

• the instance is a cluster search head.
• the search head is located in site1. A search head can belong to only one site in each cluster.
• the search head's indexer cluster manager node is located at 10.160.31.200:8089.

To disable search affinity for a search head, so that it gets its data randomly from all sites in the cluster, set the site attribute to "site0".

Note: When you specify the site parameter, the command automatically sets multisite=true in the search head's server.conf file. You do not need to explicitly pass a multisite parameter.

To edit the search head configuration later:

Use the splunk edit cluster-master command, not the splunk edit cluster-config command.

For example, assume that you initially configured a single-site search head using the splunk edit cluster-config command:

splunk edit cluster-config -mode searchhead -manager_uri https://10.160.31.200:8089  

splunk restart

To later reconfigure the search head for a multisite cluster, use the splunk edit cluster-master command:

splunk edit cluster-master https://10.160.31.200:8089  -site site1 

Important: The splunk edit cluster-master command always takes the current manager node's URI:port value as its initial parameter. For more examples, see "Configure the indexer cluster search head with the CLI".

For information on configuring a multisite search head for multi-cluster search, see "Configure multi-cluster search for multisite clusters".

Note: You do not need to restart the search head if you later change its site value.