Best practice: Forward search head data to the indexer layer

It is considered a best practice to forward all search head internal data to the search peer (indexer) layer. This has several advantages:

• It accumulates all data in one place. This simplifies the process of managing your data: You only need to manage your indexes and data at one level, the indexer level.
• It enables diagnostics for the search head if it goes down. The data leading up to the failure is accumulated on the indexers, where another search head can later access it.
• By forwarding the results of summary index searches to the indexer level, all search heads have access to them. Otherwise, they're only available to the search head that generates them.

Forward search head data

The preferred approach is to forward the data directly to the indexers, without indexing separately on the search head. You do this by configuring the search head as a forwarder. These are the main steps:

1. Make sure that all necessary indexes exist on the indexers. If you are sending internal data to any non-default index on the search head, you must add that index to the indexers. On the other hand, since, for example, the default indexes _audit and _internal exist on indexers as well as search heads, you do not need to create separate versions of those indexes to hold the corresponding search head data.

2. Configure the search head as a forwarder. Create an outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.

Here is an example outputs.conf file:

# Turn off indexing on the search head
[indexAndForward]
index = false
 
[tcpout]
defaultGroup = my_search_peers 
forwardedindex.filter.disable = true  
indexAndForward = false 
 
[tcpout:my_search_peers]
server=10.10.10.1:9997,10.10.10.2:9997,10.10.10.3:9997

This example assumes that each indexer's receiving port is set to 9997.

For details on configuring outputs.conf, read "Configure forwarders with outputs.conf" in the Forwarding Data manual.

Forward data from search head cluster members

You perform the same configuration steps to forward data from search head cluster members to their set of search peers. However, you must ensure that all members use the same outputs.conf file. To do so, do not edit the file on the individual search heads. Instead, use the deployer to propagate the file across the cluster. See "Use the deployer to distribute apps and configuration updates."