This topic covers common deployment architectures for Splunk App for Stream.
Stream deployment architectures
Splunk App for Stream supports the following deployment architectures:
- Single instance deployment
- Distributed deployment
These deployment architectures require the following Splunk Enterprise components:
- Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.
- Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.
- Universal Forwarder: A streamlined Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.
- Deployment server (optional): A Splunk Enterprise instance that deploys apps and add-ons to indexers and forwarders. You can use the deployment server to install
Splunk_TA_streamto new universal forwarders across a Stream distributed deployment.
For more information on Splunk Enterprise components, see Components of a Splunk Enterprise deployment
Single instance deployment
You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single instance deployment, which is ideal for a lab or test environment, can support one or two users running concurrent searches.
A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.
For a distributed deployment of Splunk App for Stream:
Install Splunk App for Stream (
splunk_app_stream) on one or more search heads.
Note: Splunk App for Stream does not currently support search head clustering.
Splunk_TA_stream on all indexers.
Splunk_TA_stream on any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures."
Deployment server and Distributed Management Console (DMC)
In a small, single search head deployment, both the deployment server and Distributed Management Console (DMC) components can run on the same Splunk Enterprise instance on which you install
In deployments with large numbers of forwarders (over 100), we recommend that you run
splunk_app_stream on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning."
How it works
streamfwd binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The
streamfwd binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.
Use the Streams Config UI inside
splunk_app_stream to configure the specific wire data protocols (such as http, tcp, dns) that you want the
streamfwd binary to capture.
streamfwd.xml file in
Splunk_TA_stream/local to configure system-level parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for the
streamfwd binary. See "Configure Stream forwarder".
While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install
splunk_app_stream on more than one search head.
How streamfwd communicates with splunk_app_stream
streamfwd binary pings
splunk_app_stream at regular intervals over HTTP port 8000. If
streamfwd detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated
streamfwd configuration data.
streamfwd binary pings
splunk_app_stream at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of
splunk_app_stream is stored in
$SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf. See Specify the location of splunk_app_stream.
splunk_app_stream at default intervals of 5 seconds. To change the ping interval, use the
<PingInterval> element in
streamfwd.xml. See Configure Stream forwarder.
Network collection architectures
This documentation applies to the following versions of Splunk Stream™: 6.2.0, 6.2.1, 6.2.2, 6.3.0, 6.3.1, 6.3.2