Stream forwarder sizing guide

The maximum number of Stream forwarders (streamfwd) that a search head can support depends on the value of the PingInterval in streamfwd.conf. Based on our tests, a single search running Splunk App for Stream can process approximately 150 pings + stream configuration requests per second.

Use this formula to calculate the maximum number of Stream forwarders per search head:

Stream forwarders per search head = 150 * PingInterval

The default PingInterval setting is 5 seconds, so a single search head can run up to 750 Stream Forwarders at peak activity (during startup up or when stream configuration changes are made).

Based on our tests, during average usage, a single search head running Splunk App for Stream can process approximately 400 ping requests/second. However, the startup load on the CPU is significantly higher because streamfwd must both send a ping request and pull down the full configuration from splunk_app_stream.

If your deployment includes large numbers of Stream forwarders, we suggest that you run Splunk App for Stream on a dedicated server AND stagger the startup of individual Stream forwarders over time. Starting up too many Stream forwarders simultaneously might overload the server.