Distributed Forwarder Management

Distributed Forwarder Management lets you create groups of stream forwarders with different stream capture configurations. You can use this feature to apply a specific configuration to multiple stream forwarders that have identical roles, for example specific server types (such as linux or windows) or network connection points.

Distributed Forwarder Management is useful in large scale enterprise deployments that might include 100s or 1000s of stream forwarders.

Create a stream forwarder group

To create a stream forwarder group you must specify a regex rule that matches host servers, then configure stream protocol capture for the group. You can group stream forwarders using the default Forwarder ID (which is the hostname), or specify custom Forwarder IDs to create new logical groups. You can also enable stream forwarder groups to capture ephemeral streams.

1. In the Splunk App for Stream main menu, click Distributed Forwarder Management.

This opens the Distributed Forwarder Management page, which displays your existing stream forwarder groups.

Note: If you have not yet defined a stream forwarder group, this page displays the default group, which is configured to capture all stream protocols.

2. Click Create New Group.

The Create New Forwarder Group dialog appears.

3. Enter a name and description for the group. Click Yes if you want the group to capture ephemeral streams (in addition to selected permanent streams).

4. Click Next.

The Matched Forwarders (New Rule) dialog appears.

5. Enter a regex rule. For example:

.*centos.*

A list of Forwarder IDs that match the regex rule appears in the dialog.

Note: You can change the default Forwarder ID (which is the hostname) and specify a new Forwarder ID. This lets you organize your stream forwarders into new logical groupings based on the Forwarder ID.

6. Click Next.

The Select/De-select Stream in Forwarder Group dialog appears.

7. Select the protocol that you want this group of stream forwarders to capture.

8. Click Finish.

Your new stream forwarder group appears on the Distributed Forwarder Management page.

Examples

Manage by hostname

If the existing hosts on which you install stream forwarder (Splunk_TA_stream) use a naming convention, you can create a regex rule that matches the naming convention to define a stream forwarder group.

For example, if your hosts use the naming convention my.server.01, my.server.02, my.server.03, and so on, you could use the regex *.server.* to define your stream forwarder group.

Manage by Forwarder ID

Each stream forwarder (Splunk_TA_stream) instance has its own Forwarder ID. You can change the Forwarder ID from its default value (which is the hostname) and specify a new Forwarder ID. This lets you create your own naming conventions and organize your stream forwarders into new logical groupings based on Forwarder ID.

You can specify the Forwarder ID using Splunk Web or from the command line using configuration files.

Specify a Forwarder ID using Splunk Web

1. In Splunk Web, go to Settings > Data Input > Wire Data.

2. Click on the name of the input for the specific stream forwarder.

3. In the Stream Forward Identifier field, enter a string for the Forwarder ID.

4. Click Save.

The string that you specify becomes the new Forwarder ID of the stream forwarder.

Note: When you specify the Forwarder ID using Splunk Web, you perform the configuration on the search head that hosts Splunk App for Stream. The new Forwarder ID is propagated to the corresponding stream forwarder.

Specify a forwarder ID using configuration files

1. Go to $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf

2. In the [streamfwd://streamfwd] stanza, enter a value for stream_forwarder_id:

[streamfwd://streamfwd]
splunk_stream_app_location = https://localhost:8000/en-us/custom/splunk_app_stream/
stream_forwarder_id = <Forwarder_ID>
disabled = 0

Note: When you specify the Forwarder ID using configuration files, you must perform the configuration on the forwarder(s) that host Splunk_TA_stream.