Distributed Forwarder Management

Distributed Forwarder Management lets you create groups of stream forwarders with different stream capture configurations. You can use this feature to apply a specific configuration to multiple stream forwarders that have identical roles, for example specific server types (such as Linux or Windows OS) or network connection points.

Distributed Forwarder Management is useful in large scale enterprise deployments that might include hundreds or thousands of stream forwarders.

Create a stream forwarder group

To create a stream forwarder group you specify a regex rule that matches host servers, define HTTP event collector endpoints (URLs), then select stream protocol capture for the group. You can group stream forwarders using the default Forwarder ID (which is the hostname), or specify custom Forwarder IDs to create new logical groups.

1. Click Distributed Forwarder Management.
   The Distributed Forwarder Management page displays your existing Stream forwarder groups. If you have not defined a Stream forwarder group, this page displays the default group, which is configured to capture all Stream protocols.
2. Click Create New Group.
3. Enter a name and description for the group.
4. Click Yes if you want the group to Include Ephemeral Streams (in addition to selected permanent streams).
5. Click On to enable HTTP Event Collector AutoConfig. This provides automated configuration of indexer endpoint URLs for any deployment architecture.
6. Click Next.
1. When HTTP Event Collector Autoconfig is on, splunk_app_stream automatically sends all configuration data to Stream forwarders, including endpoint URLs for all indexers (8 max). There is no need to configure data forwarding to indexers from streamfwd.
2. When HTTP Event Collector Autoconfig is off, you must manually enter the endpoint URL of each indexer for each forwarder group definition. The matching forwarder group receives the configuration from splunk_app_ stream, then sends the captured data to the specified indexers.
7. Enter a regex rule. For example: sr-centos*.*


A list of Forwarder IDs that match the regex rule appears in the dialog.

8. Click Next.
9. In the Select/De-select Stream in Forwarder Group dialog, select the protocols that you want this group of Stream forwarders to capture.
10. Click Finish.
    Your new stream forwarder group appears on the Distributed Forwarder Management page.
    You can change the default Forwarder ID (which is the hostname) and specify a new Forwarder ID. This lets you organize your stream forwarders into new logical groupings based on the Forwarder ID.

Enable HTTP event collector to send data to indexers

1. Go to Configuration > Distributed Forwarder Management.
2. Click Install Stream Forwarders.
3. If the Install Stream Forwarders modal shows that HTTP Event Collector streamfwd token configuration is disabled. Click View Configuration. The HTTP Event Collector page opens.
4. Click Global Settings, then click Enabled to enable streamfwd.
5. Make sure HTTP Port Number is set to 8088.
6. Click Save.

For information on how to configure HTTP Event Collector in an indexer cluster environment, see Propagate HTTP Event Cluster configuration to indexer cluster in the Splunk Stream Installation and Configuration Manual.

Distributed Forwarder Management Examples

Manage by hostname

If the existing hosts on which you install stream forwarder (Splunk_TA_stream) use a naming convention, you can create a regex rule that matches the naming convention to define a stream forwarder group.

For example, if your hosts use the naming convention my.server.01, my.server.02, my.server.03, and so on, you could use the regex *.server.* to define your stream forwarder group.

Manage by Forwarder ID

Each stream forwarder (Splunk_TA_stream) instance has its own Forwarder ID. You can change the Forwarder ID from its default value (which is the hostname) and specify a new Forwarder ID. This lets you create your own naming conventions and organize your stream forwarders into new logical groupings based on Forwarder ID.

You can specify the Forwarder ID using Splunk Web or from the command line using configuration files.

Specify a Forwarder ID using Splunk Web

1. In Splunk Web, go to Settings > Data Input > Wire Data.
2. Click on the name of the input for the specific stream forwarder.
3. In the Stream Forward Identifier field, enter a string for the Forwarder ID.
4. Click Save.
   The string that you specify becomes the new Forwarder ID of the stream forwarder.

Note: When you specify the Forwarder ID using Splunk Web, you perform the configuration on the search head that hosts Splunk Stream. The new Forwarder ID is propagated to the corresponding stream forwarder.

Specify a forwarder ID using configuration files

1. Go to $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf
2. In the [streamfwd://streamfwd] stanza, enter a value for stream_forwarder_id:

   [streamfwd://streamfwd]
   splunk_stream_app_location = https://localhost:8000/en-us/custom/splunk_app_stream/
   stream_forwarder_id = <Forwarder_ID>
   disabled = 0
   

Note: When you specify the Forwarder ID using configuration files, you must perform the configuration on the forwarder(s) that host Splunk_TA_stream.