Configure universal forwarder for use with Stream

Send data to _internal index

Splunk App for Stream version 6.4.1 and later automatically configures universal forwarders to send data from Splunk_TA_stream to your _internal index at installation time. If you are using Splunk App for Stream version 6.4.0 or earlier, you must manually configure universal forwarders to send data from Splunk_TA_stream to your _internal index, as follows:

1. Edit $SPLUNK_HOME/etc/system/local/outputs.conf.

2. Add the following stanza:

[tcpout]
forwardedindex.2.whitelist = (_audit|_introspection|_internal)

For more information, see Configure forwarders with outputs.conf in the Forwarding Data manual.

Manage Splunk_TA_stream stats data

Splunk_TA_stream (Stream forwarder) sends operational statistics and log data to _internal index. Splunk App for Stream uses this data to populate built-in dashboards, including the Stream Estimate dashboard.

There are three event types that Splunk_TA_stream sends to _internalindex:

• stream:stats
• stream:log
• streamfwd*

The streamfwd* event type comes from the stream forwarder's local log file. The forwarder monitors that log file along with other files in the $SPLUNK_HOME/var/log/splunk/ directory. Though it is useful for troubleshooting locally, the data that streamfwd* contains is the same as the data that stream:log contains. Therefore, you can safely blacklist the streamfwd* event type to avoid indexing duplicate data.

For more information, see Include or exclude specific incoming data in the Getting Data In manual.

Modify universal forwarder data limits

By default, the Splunk universal forwarder sends a maximum of 256 Kbps of data to your Splunk indexers. Depending on your streamfwd process throughput and configuration, Splunk_TA_stream might generate more data than this.

To modify or remove the default universal forwarder limit:

1. Edit $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf.

2. Modify the [thruput] stanza. For example:

[thruput]
maxKBps = 0