Deploy independent Stream forwarder

Splunk Stream supports independent Stream forwarder (streamfwd) installation on compatible Linux machines. To facilitate independent Stream forwarder installation, Splunk App for Stream (splunk_app_stream) generates a curl script that you can run from the command line to install Stream forwarder.

Independent Stream forwarder deployment is useful, for example, if you want to capture network data from a Linux host that you are monitoring as part of a network service in a Splunk IT Service Intelligence (ITSI) deployment.

Prerequisites

• 64-bit Linux only.
• An existing Splunk Stream 6.5.0 or later deployment.
• You must configure HTTP event collector (HEC) on indexers to receive data from independent Stream forwarder.

Independent Stream forwarder does not require Universal Forwarder.

Install independent Stream forwarder using curl

1. In the Splunk App for Stream main menu, click Configure > Distributed Forwarder Management.
2. Click Install Stream Forwarder.
   The Install Stream forwarder window appears.
3. Copy the curl script.
4. SSH into the Linux machine where you want to install Stream forwarder.
5. Run the curl script that you copied from splunk_app_stream. For example:

   curl -sSL http://stream-cont-func02:8000/en-us/custom/splunk_app_stream/install_streamfwd | sudo bash
   

6. Respond with yes or no at each prompt to download, install, and start the streamfwd binary.

Optionally, you can run the curl script in fully automated mode without prompts:

1. Run the curl script as shown in step 5 with the following parameters appended: -s -- --accept-defaults. For example:

   curl -sSL http://stream-cont-func02:8000/en-us/custom/splunk_app_stream/install_streamfwd | sudo bash -s -- --accept-defaults
   

2. Start the streamfwd service. For example:

   sudo service streamfwd start
   

After installation is complete, confirm that the splunk_stream_app_location address is set correctly in /opt/streamfwd/local/inputs.conf.

Enable SSL certificate validation

You can enable certificate validation for SSL connections to streamfwd to verify the identity of splunk_app_stream servers. For more information, see Enable TLS certificate validation in this manual.

Enable HTTP Event Collector to receive data from Stream forwarder

To receive data from independent Stream forwarder, HTTP event collector (HEC) must be enabled on Splunk indexers. There are two methods for managing HEC configuration for independent Stream forwarder:

• Use the default HEC configuration generated by splunk_app_stream on the search head.
• Manually configure streamfwd.conf on the local Stream forwarder instance.

Use default HEC configuration

When you install Splunk Stream, the app automatically generates a default HEC configuration. Independent Stream forwarders receive this default configuration from splunk_app_stream over REST API. When using the default HEC configuration, you must make sure that HEC is enabled.

1. In Splunk App for Stream, click Configuration > Distributed Forwarder Management.
2. Click Install Stream forwarders. If the HTTP Event Collector streamfwd token configuration is disabled, click View Configuration. The HTTP event collector page opens.
3. Click Global Settings.
4. In the Edit Global Setting modal, click Enabled. This enables the HTTP event collector.
5. Click Save.
6. Make sure "streamfwd" HTTP Event Collector input is present and enabled.

Use manual HEC configuration

When running independent Stream forwarder without splunk_app_stream, you must manually configure streamfwd.conf on the local Stream forwarder instance to specify the HEC token value and indexer URI.

1. Manually generate the HEC token on the indexer where you want to ingest data.
2. On the local Stream forwarder instance, edit /opt/streamfwd/local/streamfwd.conf
3. In the [streamfwd] stanza, add the following parameters to specify the HEC token value and indexer URI:

   httpEventCollectorToken = <value>
   indexer.<N>.uri = <value>
   

For instructions on generating HEC tokens, see Use HTTP Event Collector in Getting Data In.

HTTP Event Collector is supported for Independent Stream forwarder only.

Propagate HTTP Event Collector configuration to indexer cluster

HTTP Event Collector must be enabled and have the identical configuration for [httpː//streamfwd] input and SSL configuration on all indexers to which Stream forwarders are sending events.

Note that splunk_app_stream only generates the streamfwd HTTP Event Collector input on the instance on which it is running. If you want to send data to an indexer cluster, you must copy the [httpː//streamfwd] stanza from splunk_httpinput/local/inputs.conf on the appropriately configured instance, to the corresponding splunk_httpinput/local/inputs.conf files on all indexers.

For example:

[http://streamfwd]
disabled = 0
token = 521F51A6-093C-4954-80F9-47A5445DFBDD

Upgrade independent Stream forwarder

1. Log in to the Linux machine running the current version of independent Stream forwarder.
2. Backup your existing Stream forwarder configuration.
3. Stop the streamfwd service.

   cd /opt/streamfwd/bin
   sudo service streamfwd stop
   

4. Run the same curl script that you used to deploy independent Stream forwarder initially. See Install independent Stream forwarder using curl. For example:

   curl -sSL http://stream-cont-func02:8000/en-us/custom/splunk_app_stream/install_streamfwd | sudo bash
   

5. Enter [yes] at the prompt to overwrite the existing installation.
   The script overwrites the existing version of Stream forwarder with the updated version, and retains all existing Stream forwarder configurations in the migration.
6. Enter [yes] at the prompt to start the streamfwd service.