Modify the knowledge bundle

The knowledge bundle consists of a set of files that the search peers ordinarily need in order to perform their searches. You can, if necessary, modify this set of files. The main reasons for modifying the set of files are if:

• As an app developer, you want to customize the files for the needs of your app. This case usually involves manipulating the replication whitelist. You can also use a replication blacklist for this purpose.

• As an admin, you need to eliminate files from the knowledge bundle, in order to limit the bundle size. This case is somewhat unusual, because Splunk Enterprise uses delta-based replication to keep the bundle compact, with the search head usually only replicating the changed portion of the bundle to its search peers. This case requires that you identify unnecessary files and filter them out with a replication blacklist. It is also possible, although less common, to use a whitelist for this purpose.

See distsearch.conf in the Admin Manual for details on the settings discussed in this topic.

Customize the bundle for an app

The system looks at two stanzas in distsearch.conf to determine which *.conf files to include in the bundle, in this order:

1. [replicationWhitelist]

2. [replicationSettings:refineConf]

You typically only need to edit the [replicationSettings:refineConf] stanza to customize the bundle for your app, but, under rare circumstances, you might also need to modify the [replicationWhitelist] stanza.

Since the system starts by examining the [replicationWhitelist] stanza, this discussion does too.

Edit the replicationWhitelist stanza

The [replicationWhitelist] stanza in the system default version of distsearch.conf whitelists all the *.conf files that are specified in the [replicationSettings:refineConf] stanza. Therefore, to add or delete a *.conf file from the bundle, do not modify this stanza. Instead, change the set of files specified in the [replicationSettings:refineConf] stanza, as described in the next section, "Edit the replicationSettings:refineConf stanza."

The main reason for modifying the [replicationWhitelist] stanza is to include in the bundle some type of special file for use in a custom search command. This is an unusual circumstance.

If you do need to alter the whitelist, you can override the system default whitelist by creating a version of the [replicationWhitelist] stanza in $SPLUNK_HOME/etc/apps/<appname>/default/distsearch.conf:

[replicationWhitelist]
<name> = <whitelist_regex> 
...

The knowledge bundle will include all files that both satisfy the whitelist regex and are specified in [replicationSettings:refineConf]. If multiple regex's are specified, the bundle will include the union of those files.

Note: The "whitelist_regex" string for [replicationWhitelist] refers to Splunk style pattern matching, which is primarily regex-based. For details, see the entry for [replicationWhitelist] in the distsearch.conf spec file.

In this example, the knowledge bundle will include all files with extensions of either ".conf" or ".spec":

[replicationWhitelist]
allConf = *.conf
allSpec = *.spec

The names, such as allConf and allSpec, are used only for layering. That is, if you have both a global and a local copy of distsearch.conf, the local copy can be configured so that it overrides only one of the regex's. For instance, assume that the example shown above is the global copy and that you then specify a whitelist in your local copy like this:

[replicationWhitelist]
allConf = *.foo.conf

The two conf files will be layered, with the local copy taking precedence. Thus, the search head will distribute only files that satisfy these two regex's:

allConf = *.foo.conf
allSpec = *.spec

For more information on attribute layering in configuration files, see Attribute precedence in the Admin manual.

Caution: Replication whitelists are applied globally across all conf data, and are not limited to any particular app, regardless of where they are defined. Be careful to pull in only your intended files.

Edit the replicationSettings:refineConf stanza

The [replicationSettings:refineConf] stanza in distsearch.conf specifies the *.conf files and *.meta stanzas that get included in the knowledge bundle. If you want to modify the set of files in the bundle, add or delete them from this stanza.

The system default distsearch.conf file includes a version of this stanza that specifies the *.conf files that are normally included in the knowledge bundle:

[replicationSettings:refineConf]
# Replicate these specific *.conf files and their associated *.meta stanzas.
replicate.app               = true
replicate.authorize         = true
replicate.collections       = true
replicate.commands          = true
replicate.eventtypes        = true
replicate.fields            = true
replicate.segmenters        = true
replicate.literals          = true
replicate.lookups           = true
replicate.multikv           = true
replicate.props             = true
replicate.tags              = true
replicate.transforms        = true
replicate.transactiontypes  = true

If you want to replicate a .conf file that is not in the system default version of the [replicationSettings:refineConf] stanza, create a version of the stanza in $SPLUNK_HOME/etc/apps/<appname>/default/distsearch.conf and specify the *.conf file there. Similarly, you can remove files from the bundle by setting them to "false" in this stanza.

Eliminate files from the knowledge bundle

You can also create a replication blacklist, using the [replicationBlacklist] stanza. This is most useful for limiting the size of the knowledge bundle, particularly in the case of very large files that do not need to be replicated to the search peers. The blacklist takes precedence over any whitelist.

Caution: Replication blacklists are applied globally across all conf data, and are not limited to any particular app, regardless of where they are defined. If you are defining an app-specific blacklist, be careful to constrain it to match only files that your application will not need.