Configure parallel reduce search processing
To enable parallel reduce search processing for your deployment, you need to configure your indexers to work as intermediate reducers and determine how your deployment should distribute the parallel reduction workload across your indexers.
If this is your first time reading about this feature, see Overview of parallel reduce search processing for an overview of parallel reduce search processing and a list of prerequisites.
Configure your indexers to work as intermediate reducers
To gain the benefits of parallel reduce search processing, you must configure all of your indexers so that they have the potential to work as intermediate reducers. You accomplish this configuration by giving each of your indexers an identical pass4SymmKey
security key. This security key enables secure communication between indexers and intermediate reducers.
To update your indexer configurations, you must have access to the server.conf
file for your Splunk deployment, located in $SPLUNK_HOME/etc/system/local/
. See About configuration files and the topics that follow it in the Admin Manual for more information about making configuration file updates.
Parallel reduce search processing is not site-aware. Do not add this configuration to your indexers if they are in a multisite indexer cluster or if they are non-clustered and spread across several sites.
Set a security key for your intermediate reducers
Place a pass4SymmKey
security key in a [parallelreduce]
stanza for each indexer configuration in server.conf
. The security key value must be identical for each indexer. It secures communication between the indexers and the intermediate reducers in your deployment.
Your indexer configurations might already have pass4SymmKey
values under their [general]
and [clustering]
stanzas. Do not change those pass4SymmKey
settings. Do not use the same security key values as those pass4SymmKey
settings.
Save a copy of the key. After you set the key for an indexer and reboot the indexer, the security key changes from clear text to encrypted form, and it is no longer recoverable from server.conf
. If you add a new intermediate reducer later, you must use the clear text version of the key to set it.
Prerequisites
The following prerequisite topics are useful if you run an indexer cluster.
- Secure your clusters with pass4SymmKey, in Securing Splunk Enterprise. Learn how
pass4SymmKey
is also used to authenticate communications between members of indexer clusters and search head clusters. - Configure the indexer cluster with server.conf and Configure peer nodes with server.conf, in Managing Indexers and Clusters of Indexers. Learn how to update configurations for individual indexers.
Steps
- Open
server.conf
and locate the settings for an indexer. Indexers are identified with a[<hostname>:<port>]
stanza. - Add the following stanza and security configuration to the settings for the indexer:
[parallelreduce] pass4SymmKey=<password>
- Save your
server.conf
changes. - Restart the indexer with the CLI
restart
command:$SPLUNK_HOME/bin/splunk restart
Repeat these steps for each indexer in your deployment. Use the same <password>
for each indexer in your deployment.
Determine how your parallel reduction workload is distributed
Settings in the [parallelreduce]
stanza of limits.conf
determine the number of intermediate reducers that are selected from your indexers for a parallel reduce search process. They also determine how parallel reduce search processing work is distributed across your indexers.
For example, if you keep the default parallel reduce settings in limits.conf
, the Splunk platform randomly selects a certain number of intermediate reducers each time you run a parallel reduce search. If all of your indexers are in a single-site indexer cluster, the random selection aids in distributing the parallel reduction workload across the cluster.
However, if your indexers are not clustered, and some of your indexers have large indexing loads on average while others do not, you can use the reducers
setting to configure the low-load indexers to be dedicated intermediate reducers. Dedicated intermediate reducers are always used when you run a parallel reduce search process.
These two methods are mutually exclusive. When you set up dedicated intermediate reducers, the Splunk platform cannot randomly select intermediate reducers.
To configure parallel reduce search processing, you must have access to the limits.conf
file for your Splunk deployment, located in $SPLUNK_HOME/etc/system/local/
. See About configuration files and the topics that follow it in the Admin Manual for more information about making configuration file updates.
Enable random selection of intermediate reducers
Random selection of indexers for intermediate reduction service is ideal if you are running a single-site indexer cluster. If you run several parallel reduce searches concurrently, the random selection ensures that the intermediate reduction work is evenly distributed across the cluster.
The default parallel reduce search processing settings enable the Splunk platform to randomly select intermediate reducers from the larger set of indexers when you run parallel reduce searches. The default number of indexers that the Splunk platform repurposes as intermediate reducers during the intermediate reduce phase of the parallel reduce search process is 50% of the total number of indexers in your indexer pool, up to a maximum of 4 indexers.
Random intermediate reducer selection is determined by the maxReducersPerPhase
and winningRate
settings. They belong to the [parallelreduce]
stanza of limits.conf
.
Setting name | Definition | Default value |
---|---|---|
maxReducersPerPhase
|
The maximum number of indexers that can be used as intermediate reducers in the intermediate reduce phase of a parallel reduce search. | 4 |
winningRate
|
The percentage of indexers that can be selected from the total pool of indexers and used as intermediate reducers in a parallel reduce search process. This setting applies only when the reducers setting is not configured in limits.conf . See Enable dedicated intermediate reducers.
|
50 |
Enable dedicated intermediate reducers
To configure a set of non-clustered indexers as dedicated intermediate reducers, add the reducers
setting to the [parallelreduce]
stanza in limits.conf
.
The value of reducers
is a comma-separated list of indexers that you have configured as search peers. Identify each indexer by specifying its host and port using the following format: <host>:<port>
. For example:
reducers=docteam-unix-4:8089, docteam-unix-5:8089, docteam-unix-6:8089
Do not include clustered indexers on the reducers
list.
All indexers in the reducers
list are used as intermediate reducers when you run a parallel reduce search. If the number of indexers in the reducers
list exceeds the value of the maxReducersPerPhase
setting, the Splunk platform randomly selects the intermediate reducers from the reducers
list. For example, if the reducers
setting lists five reducers and maxReducersPerPhase=4
, the Splunk platform randomly selects four intermediate reducers from the list.
If all of the indexers in the reducers
list are down or are otherwise invalid, searches with the redistribute
command run without parallel reduction. All reduce operations are processed on the search head.
When you configure the reducers
setting for your deployment, the Splunk platform ceases to apply the winningRate
setting.
Override the number of reducers for a specific search
When you run a parallel reduce search with the redistribute
command, you can use the num_of_reducers
argument to override the number of reducers determined by the parallel reduce search settings in the limits.conf
file.
For example, say your limits.conf
settings determine that seven intermediate reducers are used by default in all parallel reduce searches. You can design a parallel reduce search where num_of_reducers = 5
. Every time that search runs, only five intermediate reducers are used in its intermediate reduce phase.
If you provide a value for the num_of_reducers
setting that exceeds the limit set by the maxReducersPerPhase
setting in the limits.conf
file, the Splunk platform sets the number of reducers to the maxReducersPerPhase
value.
Next steps
Use the redistribute
command to apply parallel reduce search processing to your high-cardinality searches. See Apply parallel reduce processing to searches.
Overview of parallel reduce search processing | Apply parallel reduce processing to searches |
This documentation applies to the following versions of Splunk® Enterprise: 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10
Feedback submitted, thanks!