Splunk Stream deployment architectures

Splunk Stream deployments require installation of Stream components on existing Splunk Enterprise instances and/or compatible Linux machines. Splunk Stream includes three components, including splunk_app_stream, Splunk_TA_stream, and independent Stream forwarder (streamfwd binary). For details on Splunk Stream components, see About the Splunk Stream Installation Package in this manual.

Splunk Stream supports these deployment architectures:

• Single instance deployment
• Distributed deployment
• Independent Stream forwarder deployment

Single instance deployment

You can install Splunk Stream on a single Splunk Enterprise instance. A single instance serves as both search head and indexer, providing both search and storage capability. A single instance deployment can support one or two users running concurrent searches, which is ideal for a small test environment. For single instance installation instructions, see Install Splunk Stream on a single instance in this manual.

Distributed deployment

A Splunk Stream distributed deployment can capture network event data from multiple network devices, including NICs, switches, routers, and so on. A distributed deployment can apply to many types of medium and large enterprise network infrastructures. For distributed installation instructions, see Install Splunk Stream in a distributed environment in this manual.

The diagram shows a typical Splunk Stream distributed deployment architecture.

A Splunk Stream distributed deployment includes the following components:

Search Heads

splunk_app_stream and Splunk_TA_stream are required on search heads.

Indexers

Splunk_TA_stream is required on all indexers for searching and parsing. The TA contains both search and index time props. splunk_app_stream is not required on indexers.

Universal forwarders

Splunk_TA_stream is required on universal forwarders at the location(s) where you want to capture network data. For more information, see Network collection architectures in this manual.

Deployment server

Use the Splunk deployment server to distribute Splunk_TA_stream to universal forwarders across a distributed deployment. When you upgrade to a new version of Splunk Stream, if the deployment server detects a new version of Splunk_TA_stream, all universal forwarders subscribed as deployment clients will pull and install the new version of the TA. For more information, see Deployment server provisioning in Upgrading Splunk Enterprise Instances.

For more information, see Components of a Splunk Enterprise deployment in the Splunk Enterprise Capacity Planning Manual.

How a distributed Splunk Stream deployment works

In a typical distributed deployment, Spunk_TA_stream, which includes the streamfwd binary, is installed on universal forwarders and captures network event data on local NICs (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures in this manual.

The actual network data that streamfwd captures depends on the specific protocols and fields that you select when you configure a stream using the Configure Streams UI inside the app. Splunk_TA_stream then sends captured event data to indexers using the pre-enabled Wire Data modular input.

How streamfwd communicates with splunk_app_stream

The streamfwd binary pings splunk_app_stream at default 5 second intervals over HTTP port 8000. If streamfwd detects a change in stream configuration (set in the Configure Streams UI), it sends an API request to the endpoint to get the latest configuration data. The location of splunk_app_stream is stored in Splunk_TA_stream/local/inputs.conf. For more information, see Specify the location of splunk_app_stream.

Independent Stream forwarder deployment

In an independent Stream forwarder deployment, you install the Stream forwarder (streamfwd) binary directly on a Linux machine. Splunk Stream generates a curl command that you can copy and run from the command line on the target machine to install the Stream forwarder component.

Independent Stream forwarder deployment can be useful, for example, if you want to monitor activities on a single Linux host that is part of a network service in a Splunk IT Service Intelligence (ITSI) deployment. No Splunk platform components are required on the Linux host.

For independent Stream forwarder installation instructions, see Install independent Stream forwarder in this manual.

How an independent Stream forwarder deployment works

Unlike typical Splunk Stream distributed deployments, independent Stream forwarder deployments do not require universal forwarders to send data to indexers. Independent Stream forwarder deployments send captured event data to indexers using the HTTP event collector. For more information, see Deploy Independent Stream forwarder in this manual.