Configure the site search factor
Read this first
Before attempting to configure the site search factor, you must be familiar with:
- The basic, single-site search factor. See "The basics of cluster architecture" and "Search factor".
- The site replication factor. See "Configure the site replication factor".
- Multisite cluster configurations. See "Configure multisite indexer clusters with server.conf".
What is a site search factor?
To implement multisite indexer clustering, you must configure a site search factor. This replaces the standard search factor, which is specific to single-site deployments. You specify the site search factor on the manager node, as part of the basic configuration of the cluster.
The site search factor provides site-level control over the location of searchable bucket copies, in addition to providing control over the total number of searchable copies across the entire cluster. For example, you can specify that a two-site cluster maintain a total of three searchable copies of all buckets, with one site maintaining two copies and the second site maintaining one copy.
You can also specify a search policy based on which site originates the bucket. That is, you can configure the search factor so that a site receiving external data maintains a greater number of searchable copies of buckets for that data than for data that it does not originate. For example, you can specify that each site maintains two searchable copies of all data that it originates but only one copy of data originating on another site.
The site search factor helps determine whether the cluster has search affinity. See "Implement search affinity in a multisite indexer cluster".
Syntax
The syntax for site_search_factor
and site_replication_factor
are identical, except for the additional requirement that the values and explicit sites in site_search_factor
be a subset of those in site_replication_factor
. This section describes the syntax in full detail.
You configure the site search factor with the site_search_factor
attribute in the manager's server.conf file. The attribute resides in the [clustering]
stanza, in place of the single-site search_factor
attribute. For example:
[clustering] mode = manager multisite=true available_sites=site1,site2 site_replication_factor = origin:2,total:3 site_search_factor = origin:1,total:2
You can also use the CLI to configure the site search factor. See "Configure multisite indexer clusters with the CLI".
Warning: You must configure the site_search_factor
attribute correctly. Otherwise, the manager will not start.
Here is the formal syntax:
site_search_factor = origin:<n>, [site1:<n>,] [site2:<n>,] ..., total:<n>
where:
<n>
is a positive integer indicating the number of searchable copies of a bucket.origin:<n>
specifies the minimum number of searchable copies of a bucket that will be held on the site originating the data in that bucket (that is, the site where the data first entered the cluster). When a site is originating the data, it is known as the "origin" site.site1:<n>, site2:<n>, ...,
indicates the minimum number of searchable copies that will be held at each specified site. The identifiers "site1", "site2", and so on, are the same as thesite
attribute values specified on the peer nodes.total:<n>
specifies the total number of searchable copies of each bucket, across all sites in the cluster.
Note the following:
- This attribute specifies the per-site searchable copy policy. It is specified globally and applies to all buckets in all indexes.
- This attribute is valid only if
mode=manager
andmultisite=true
. Under those conditions, it supersedes anysearch_factor
attribute.
- The
origin
andtotal
values are required.
- Site values (
site1:<n>, site2:<n>, ...
) are optional. A site that is specified here is known as an "explicit" site. A site that is not specified is known as a "non-explicit" site.
- To determine the minimum number of searchable copies a site gets, use the same rules as for determining the minimum number of replicated copies a site gets through
site_replication_factor
. See "Configure the site replication factor".
- To determine the minimum required value for
total
, use the same rules as for determining the minimumtotal
value for thesite_replication_factor
. See "Configure the site replication factor".
- Because the
total
value can be greater than the total set of explicit values, the cluster needs a strategy to handle any "remainder" searchable bucket copies. The strategy follows the strategy for remainder replicated copies, described in "Configure the site replication factor".
- All values must be less than or equal to their corresponding values in the
site_replication_factor
.
For example, if you have a three-site cluster with "site_replication_factor = origin:2, site1:1, site2:2, total:5", then, insite_search_factor
, theorigin
value cannot exceed 2, thesite1
value cannot exceed 1, thesite2
value cannot exceed 2, and thetotal
value cannot exceed 5.
- If a site value is explicit in
site_search_factor
, it must also be explicit insite_replication_factor
. However, an explicit site value insite_replication_factor
does not need be explicit insite_search_factor
.
For example, if you have a three-site cluster with "site_replication_factor = origin:2, site1:1, site2:2, total:5" (with a non-explicit site3), you can specify "site_search_factor = origin:1, site2:2, total:4" (removing the explicit site1), but you cannot specify "site_search_factor = origin:1, site1:1, site2:2, site3:1, total:4" (making the non-explicit site3 explicit).
- For search affinity, you must configure the
site_search_factor
so that you have at least one searchable copy on each site where you require search affinity. Only explicit sites adhere to search affinity.
- If you are migrating from a single-site cluster, the
total
value must be at least as large as thesearch_factor
for the single-site cluster. See "Migrate an indexer cluster from single-site to multisite".
- The attribute defaults to: "origin:1, total:2."
Examples
For examples of site search factor syntax, refer to the examples in "Configure the site replication factor". The syntax for specifying origin/site/total values in site_search_factor
is identical to site_replication_factor
.
Handle the persistence of the single-site search_factor
Important: Although the site_search_factor
effectively replaces the single-site search_factor
, the single-site attribute continues to exist in the manager's configuration, with its default value of 2. This can cause problems on start-up if any site has only one peer node. The symptom will be a message that the multisite cluster does not meet its search factor. To avoid or fix this problem, you must set the single-site search factor to a value that is less than or equal to the smallest number of peers on any site. In the case where one site has only one peer, you must therefore explicitly set the search_factor
attribute to a value of 1. See "Multisite cluster does not meet its replication or search factors."
Configure the site replication factor | Migrate an indexer cluster from single-site to multisite |
This documentation applies to the following versions of Splunk® Enterprise: 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.3.0, 9.3.1
Feedback submitted, thanks!