Modify the knowledge bundle
The knowledge bundle is the data that the search head replicates and distributes to each search peer to enable its searches. For information on the contents and purpose of this bundle, see "What search heads send to search peers".
The knowledge bundle consists of a set of files that the search peers ordinarily need in order to perform their searches. You can, if necessary, modify this set of files. The main reasons for modifying the set of files are if:
- As an app developer, you want to customize the files for the needs of your app. This case usually involves manipulating the replication whitelist. You can also use a replication blacklist for this purpose.
- As an admin, you need to limit the size of the knowledge bundle. This case is somewhat unusual, because Splunk Enterprise uses delta-based replication to keep the bundle compact, with the search head usually only replicating the changed portion of the bundle to its search peers. This case requires that you identify unnecessary files and filter them out with a replication blacklist. It is also possible, although less common, to use a whitelist for this purpose.
See distsearch.conf in the Admin Manual for details on the settings discussed in this topic.
Customize the bundle for an app
The system looks at two stanzas in
distsearch.conf to determine which
*.conf files to include in the bundle, in this order:
You typically only need to edit the
[replicationSettings:refineConf] stanza to customize the bundle for your app, but, under rare circumstances, you might also need to modify the
Since the system starts by examining the
[replicationWhitelist] stanza, this discussion does too.
Edit the replicationWhitelist stanza
[replicationWhitelist] stanza in the system default version of
distsearch.conf whitelists all the
*.conf files that are specified in the
[replicationSettings:refineConf] stanza. Therefore, to add or delete a
*.conf file from the bundle, do not modify this stanza. Instead, change the set of files specified in the
[replicationSettings:refineConf] stanza, as described in the next section, "Edit the replicationSettings:refineConf stanza."
The main reason for modifying the
[replicationWhitelist] stanza is to include in the bundle some type of special file for use in a custom search command. This is an unusual circumstance.
If you do need to alter the whitelist, you can override the system default whitelist by creating a version of the
[replicationWhitelist] stanza in
[replicationWhitelist] <name> = <whitelist_regex> ...
The knowledge bundle will include all files that both satisfy the whitelist regex and are specified in
[replicationSettings:refineConf]. If multiple regex's are specified, the bundle will include the union of those files.
In this example, the knowledge bundle will include all files with extensions of either ".conf" or ".spec":
[replicationWhitelist] allConf = *.conf allSpec = *.spec
The names, such as allConf and allSpec, are used only for layering. That is, if you have both a global and a local copy of
distsearch.conf, the local copy can be configured so that it overrides only one of the regex's. For instance, assume that the example shown above is the global copy and that you then specify a whitelist in your local copy like this:
[replicationWhitelist] allConf = *.foo.conf
The two conf files will be layered, with the local copy taking precedence. Thus, the search head will distribute only files that satisfy these two regex's:
allConf = *.foo.conf allSpec = *.spec
For more information on attribute layering in configuration files, see "Attribute precedence" in the Admin manual.
Caution: Replication whitelists are applied globally across all conf data, and are not limited to any particular app, regardless of where they are defined. Be careful to pull in only your intended files.
Edit the replicationSettings:refineConf stanza
[replicationSettings:refineConf] stanza in
distsearch.conf specifies the
*.conf files and
*.meta stanzas that get included in the knowledge bundle. If you want to modify the set of files in the bundle, add or delete them from this stanza.
The system default
distsearch.conf file includes a version of this stanza that specifies the
*.conf files that are normally included in the knowledge bundle:
[replicationSettings:refineConf] # Replicate these specific *.conf files and their associated *.meta stanzas. replicate.app = true replicate.authorize = true replicate.collections = true replicate.commands = true replicate.eventtypes = true replicate.fields = true replicate.segmenters = true replicate.literals = true replicate.lookups = true replicate.multikv = true replicate.props = true replicate.tags = true replicate.transforms = true replicate.transactiontypes = true
If you want to replicate a
.conf file that is not in the system default version of the
[replicationSettings:refineConf] stanza, create a version of the stanza in
$SPLUNK_HOME/etc/apps/<appname>/default/distsearch.conf and specify the
*.conf file there. Similarly, you can remove files from the bundle by setting them to "false" in this stanza.
Limit the size of the knowledge bundle
You can also create a replication blacklist, using the
[replicationBlacklist] stanza. This is most useful for limiting the size of the knowledge bundle, particularly in the case of very large files that do not need to be replicated to the search peers. The blacklist takes precedence over any whitelist.
Caution: Replication blacklists are applied globally across all conf data, and are not limited to any particular app, regardless of where they are defined. If you are defining an app-specific blacklist, be careful to constrain it to match only files that your application will not need.
What search heads send to search peers
About mounted bundles
This documentation applies to the following versions of Splunk® Enterprise: 6.5.7, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13
Feedback submitted, thanks!