Deploy a search head cluster in a multisite environment

You can deploy search head cluster members across multiple physical sites. You can also integrate cluster members into a multisite indexer cluster. However, search head clusters do not have site awareness.

Deploy a search head cluster across multiple physical sites

There are no restrictions on where your cluster members can reside. In cases of high network latency between sites, however, you might notice some slowness in UI responsiveness.

The amount of data that cluster members transfer to each other across the network is difficult to quantify, being dependent on a variety of factors, such as the number of users, the amount of user activity, the number and types of searches being run, and so on.

Integrate a search head cluster with a multisite indexer cluster

You can integrate the search head cluster members into a multisite indexer cluster. A multisite indexer cluster confers important advantages on your deployment. Most importantly, it enhances the high availability and disaster recoverability of your deployment. See "Multisite indexer clusters" in the Managing Indexers and Clusters manual.

To integrate a search head cluster with a multisite indexer cluster, configure each member as a search head in the multisite cluster. See "Integrate with a multisite indexer cluster."

It is recommended that you set each search head's site attribute to "site0", to disable search affinity. When search affinity is disabled, the search head runs its searches across indexers spanning all sites. Barring any change in the set of available indexers, the search head will run its searches across the same set of primary bucket copies each time.

By setting all search heads to "site0", you ensure a seamless experience for end users, because the same set of primary bucket copies is used by all search heads. If, instead, you set different search heads to different sites, the end user might notice lag time in getting some results, depending on which search head happens to run a particular search.

If you have an overriding need for search affinity, you can assign the search heads to specific sites.

Search head clusters do not have site awareness

Unlike an indexer cluster, search head clusters lack site awareness:

• You cannot configure artifact replication on a site-by-site basis.
• The cluster does not guarantee that copies of each search artifact exist on each site.

Site awareness is less critical for a search head cluster than an indexer cluster. If a search head cluster member is missing a replicated copy of a search artifact, the cluster proxies it from another member, which could reside on the same site or on another site. See "How the cluster handles search artifacts." Even in the case of a site failure that results in the loss of all copies of some search artifacts, this is a manageable situation that you can recover from by rerunning searches and so on.

Note: There are ways that you can work around the lack of site awareness, if necessary. For example, if your search head cluster consists of four search heads divided evenly between two sites, you can set the replication factor to 3 and thus ensure that each site has at least one copy of each search artifact.

Important considerations when deploying a search head cluster across multiple sites

The choices you make when deploying a search head cluster across multiple sites can have significant implications for these failure scenarios:

• Site failure
• Network interruptions

In particular, in the case of a two-site cluster, you should put the majority of your members on the site that you consider primary.

Why the majority of members should be on the primary site

If you are deploying the cluster across two sites, put a majority of the cluster members on the site that you consider primary. This ensures that the cluster can continue to function as long as that site is running.

Under certain circumstances, such as when a member leaves or joins the cluster, the cluster holds an election in which it chooses a new captain. The success of this election process requires that a majority of all cluster members agree on the new captain. Therefore, the proper functioning of the cluster requires that a majority of members be running at all times. See "Captain election."

In the case of a cluster running across two sites, if one site fails, the remaining site can elect a new captain only if it holds a majority of members. Similarly, if there is a network disruption between the sites, only the site with a majority can elect a new captain. By assigning the majority of members to your primary site, you maximize its availability.

What happens when the site with the majority fails

If the site with a majority of members fails, the remaining members on the minority site cannot elect a new captain. Captain election requires the vote of a majority of members, but only a minority of members are running. The cluster does not function. See "Consequences of a non-functioning cluster."

To remediate this situation, you can temporarily deploy a static captain on the minority site. Once the majority site returns, you should revert the minority site to the dynamic captain. See "Use static captain to recover from loss of majority."

What happens when there is a network interruption between sites

If the network between sites fails, the members on each site will attempt to elect a captain. However, only a site that holds a majority of the total members will succeed. That site can continue to function as the cluster indefinitely.

During this time, the members on the other sites can continue to function as independent search heads. However, they will only be able to service ad hoc searches. Scheduled reports and alerts will not run, because, in a cluster, the scheduling function is relegated to the captain.

When the other sites reconnect to the majority site, their members will rejoin the cluster. For details on what happens when a member rejoins the cluster, see "When the member rejoins the cluster."

Clusters with more than two sites

If there are more than two sites, the cluster can function only if a majority of members across the sites are still able to communicate and elect a captain. For example, if you have site1 with five members, site2 with eight members, and site3 with four members, the cluster can survive the loss of any one site, because you will still have a majority of members (at least nine) among the remaining two sites. However, if you have site1 with six members, site2 with two members, and site3 with three members, the cluster can only function as long as site1 remains alive, because you need at least six members to constitute a majority.