Forwarder-to-indexer ratios

Splunk Enterprise indexers are responsible for accepting data streams from internal and external sources, such as forwarders, and indexing that stream locally. Indexing the data requires plentiful disk I/O bandwidth and some computing resources. Indexing capacity remains the top concern when you consider how many forwarders an indexer can handle.

The number of forwarders from which an indexer can accept data depends on several factors:

• Number of CPU cores on the machine. The number of cores should meet or exceed the reference standard.
• The storage available to the machine should meet or exceed the reference standard.
• Whether the indexer runs Windows or *nix.
• The amount of data to be forwarded to the indexers.
• Whether the indexer also acts as a deployment server.

Forwarder-to-indexer ratio testing for a *nix instance

To provide guidance for the estimated number of forwarders that can connect to a single *nix instance of Splunk Enterprise, a test was setup with:

• A Splunk Enterprise instance with 8 cores and 7GB of RAM and 4 x 420GB disks in RAID 0, running a 64-bit Linux OS.
• A high-speed local area network (LAN) operating at 100Mb/s or faster.
• A pool of universal forwarders sending data that was not pre-processed.

In these circumstances, the instance was able to handle a minimum of 2000 forwarders and regularly handled as many as 5000 forwarders.

Performance was best when the server was configured to accept a high number of Unix file descriptors, typically three to four times the number of forwarders that the indexer could accept.

Note: These numbers are for guidance only. Results vary depending on the configuration of the indexers, forwarders, and network.