Deployment architectures
This topic covers common deployment architectures for Splunk App for Stream.
Splunk App for Stream components
Splunk App for Stream includes the following components:
- Splunk Stream Add-on (Splunk_TA_stream):
Splunk_TA_stream
captures network event data, and sends that data to Splunk indexers.Splunk_TA_stream
contains thestreamfwd
binary (also called Stream Forwarder), which passively captures event data from network devices, and runs a Wire Data modular input that sends event data to STDOUT using XML streaming mode.
- Splunk App for Stream (splunk_app_stream):
splunk_app_stream
provides configuration management of network protocol events that thestreamfwd
binary captures. See Supported Protocols.splunk_app_stream
also provides pre-built dashboards that display network interface andstreamfwd
binary process metrics.
Common deployment architectures
The most common Splunk App for Stream deployment architectures include:
- Single instance deployment
- Distributed deployment
These deployment architectures require the following Splunk Enterprise components:
- Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.
- Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.
- Forwarder: A Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.
Single instance deployment
You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single server configuration, which is ideal for a lab or test environment, can support one or two users running concurrent searches.
Distributed deployment
A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.
For a distributed deployment of Splunk App for Stream:
- Install Splunk App for Stream (
splunk_app_stream
) on a Splunk Enterprise instance (6.0 or later).
- Install
Splunk_TA_stream
on any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures".
In a small, single search head deployment, both the Distributed Management Console (DMC) and deployment server components can run on the same Splunk Enterprise instance on which you install splunk_app_stream
.
In deployments with large numbers of forwarders (over 100), we recommend that you run splunk_app_stream
on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning".
How it works
The streamfwd
binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The streamfwd
binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.
There are two types of configuration that apply to Splunk App for Stream:
- Streams Config UI: Use the Streams Config UI in
splunk_app_stream
(on the search head) to configure the specific wire data protocols (such as http, tcp, dns) that you want thestreamfwd
binary to capture.
- streamfwd.xml: Use the
streamfwd.xml
file inSplunk_TA_stream/local
(on the universal forwarder) to configure parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for thestreamfwd
binary. See "Configure Stream forwarder".
While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install splunk_app_stream
on more than one search head.
About Search Head Clustering
Splunk App for Stream is not compatible with search head clustering. As a workaround, you can choose one node of a search head cluster as the Stream configuration and management server. Once you install and configure Splunk App for Stream on the single search cluster node, all Stream Add-on forwarders must point to that search cluster node. You must also perform all stream capture configuration on the search cluster node that hosts the Splunk App for Stream.
How streamfwd communicates with splunk_app_stream
The streamfwd
binary pings splunk_app_stream
at regular intervals over HTTP port 8000. If streamfwd
detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated streamfwd
configuration data.
The streamfwd
binary pings splunk_app_stream
at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of splunk_app_stream
is stored in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf
. See Specify the location of splunk_app_stream.
Note: streamfwd
pings splunk_app_stream
at default intervals of 5 seconds. To change the ping interval, use the <PingInterval>
element in streamfwd.xml
. See Configure Stream forwarder.
Deployment requirements | Network collection architectures |
This documentation applies to the following versions of Splunk Stream™: 6.0, 6.0.1, 6.0.2, 6.1.0, 6.1.1
Feedback submitted, thanks!