Auto load balancing

Load balancing is a technique for distributing a data stream across multiple receiving Splunk indexers in a predefined group. When combined with distributed search, it allows you to scale your Splunk environment horizontally to accommodate greater amounts of data with improved search performance.

To perform load balancing, you configure a Splunk forwarder to distribute data among a set of indexers named in a target group. The forwarder forwards data in sequence. It sends data first to indexer "A", then it switches to indexer "B", then to indexer "C", then back to indexer "A", indexer "B", and so on. Each indexer receives a slice of the data.

Together, the load-balanced group holds the full data stream. If a server goes down, the forwarder immediately begins distributing data to the next server in the sequence, providing a measure of fault tolerance. In automatic load balancing, the forwarder uses a specified time interval to determine when to begin sending data to the next receiving indexer.

To set up auto load balancing

You can set up auto load balancing by editing the outputs.conf file as shown in the following example:

To enable auto load balancing in the outputs.conf file, you set autoLB to true and define a default group of indexers to receive events from the forwarder by specifying a group name, such as indexers1. Your outputs.conf file may look resemble this:

[tcpout] 
defaultGroup=indexers1 
autoLB=true 
disabled=false 

[tcpout:indexers1] 
server=<ipaddressofindexer1>:9997,<ipaddressofindexer2>:9997 
autoLB=true 
disabled=false

On the "server" line: specify the list of Splunk indexers that you want to auto load balance across. While only two servers are shown in the template, the line typically lists of all of your Splunk indexers. You can add as many indexers to the file as you need. The format of each server is an IP address (or DNS hostname) and the port number you configured your indexers to receive data on. The template shows port 9997 as that is the default port that Splunk uses, unless otherwise specified.

For example, assume you have 2 indexers located at IP addresses 10.1.8.27 and 10.1.8.28 that are configured to use the default Splunk port 9997. Using the template, this is how your outputs.conf file will look:

[tcpout:indexers1] 
server=10.1.8.27:9997, 10.1.8.28:9997 
autoLB=true 
disabled=false

The forwarder will load balance between the two indexers listed. If one indexer goes down, the forwarder automatically switches to the next one available.

• For information about setting up load balancing, see "Set up load balancing" in the Splunk Distributed Deployment Guide.
• For more information about load balancing in outputs.conf, see "Configure forwarders with outputs.conf" as described in relation to typical deployment topologies and "Forwarder deployment topologies" in the Splunk Distributed Deployment Manual.