This topic covers common deployment architectures for Splunk App for Stream.
Splunk App for Stream components
Splunk App for Stream includes the following components:
- Splunk Stream Add-on (Splunk_TA_stream):
Splunk_TA_streamcaptures network event data, and sends that data to Splunk indexers.
streamfwdbinary (also called Stream Forwarder), which passively captures event data from network devices, and runs a Wire Data modular input that sends event data to STDOUT using XML streaming mode.
- Splunk App for Stream (splunk_app_stream):
splunk_app_streamprovides configuration management of network protocol events that the
streamfwdbinary captures. See Supported Protocols.
splunk_app_streamalso provides pre-built dashboards that display network interface and
streamfwdbinary process metrics.
Common deployment architectures
The most common Splunk App for Stream deployment architectures include:
- Single instance deployment
- Distributed deployment
These deployment architectures require the following Splunk Enterprise components:
- Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.
- Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.
- Forwarder: A Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.
Single instance deployment
You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single server configuration, which is ideal for a lab or test environment, can support one or two users running concurrent searches.
A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.
For a distributed deployment of Splunk App for Stream:
- Install Splunk App for Stream (
splunk_app_stream) on a Splunk Enterprise instance (6.0 or later).
Splunk_TA_streamon any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures".
In a small, single search head deployment, both the Distributed Management Console (DMC) and deployment server components can run on the same Splunk Enterprise instance on which you install
In deployments with large numbers of forwarders (over 100), we recommend that you run
splunk_app_stream on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning".
How it works
streamfwd binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The
streamfwd binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.
There are two types of configuration that apply to Splunk App for Stream:
- Streams Config UI: Use the Streams Config UI in
splunk_app_stream(on the search head) to configure the specific wire data protocols (such as http, tcp, dns) that you want the
streamfwdbinary to capture.
- streamfwd.xml: Use the
Splunk_TA_stream/local(on the universal forwarder) to configure parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for the
streamfwdbinary. See "Configure Stream forwarder".
While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install
splunk_app_stream on more than one search head.
About Search Head Clustering
Splunk App for Stream is not compatible with search head clustering. As a workaround, you can choose one node of a search head cluster as the Stream configuration and management server. Once you install and configure Splunk App for Stream on the single search cluster node, all Stream Add-on forwarders must point to that search cluster node. You must also perform all stream capture configuration on the search cluster node that hosts the Splunk App for Stream.
How streamfwd communicates with splunk_app_stream
streamfwd binary pings
splunk_app_stream at regular intervals over HTTP port 8000. If
streamfwd detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated
streamfwd configuration data.
streamfwd binary pings
splunk_app_stream at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of
splunk_app_stream is stored in
$SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf. See Specify the location of splunk_app_stream.
splunk_app_stream at default intervals of 5 seconds. To change the ping interval, use the
<PingInterval> element in
streamfwd.xml. See Configure Stream forwarder.
Network collection architectures
This documentation applies to the following versions of Splunk Stream™: 6.0, 6.0.1, 6.0.2, 6.1.0, 6.1.1