Best practice: Forward search head data to the indexer layer
It is considered a best practice to forward all search head internal data to the search peer (indexer) layer. This has several advantages:
- It accumulates all data in one place. This simplifies the process of managing your data: You only need to manage your indexes and data at one level, the indexer level.
- It enables diagnostics for the search head if it goes down. The data leading up to the failure is accumulated on the indexers, where another search head can later access it.
- By forwarding the results of summary index searches to the indexer level, all search heads have access to them. Otherwise, they're only available to the search head that generates them.
The preferred approach is to forward the data directly to the indexers, without indexing separately on the search head. You do this by configuring the search head as a forwarder. These are the main steps:
1. Make sure that all necessary indexes exist on the indexers. For example, the S.o.S app uses a scripted input that puts data into a custom index. If you install S.o.S on the search head, you need to also install the S.o.S Add-on on the indexers, to provide the indexers with the necessary index settings for the data the app generates. On the other hand, since
_internal exist on indexers as well as search heads, you do not need to create separate versions of those indexes to hold the corresponding search head data.
2. Configure the search head as a forwarder. Create an
outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.
Here is an example
# Turn off indexing on the search head [indexAndForward] index = false [tcpout] defaultGroup = my_search_peers forwardedindex.filter.disable = true indexAndForward = false [tcpout:my_search_peers] server=10.10.10.1:9997,10.10.10.2:9997,10.10.10.3:9997 autoLB = true
This assumes that each indexer's receiving port is set to 9997.
For details on configuring
outputs.conf, read "Configure forwarders with outputs.conf" in the Forwarding Data manual.
Manage distributed server names
About mounted bundles
This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14