Deployment architectures
This topic covers common deployment architectures for Splunk App for Stream.
Stream deployment architectures
Splunk App for Stream supports the following deployment architectures:
- Single instance deployment
- Distributed deployment
These deployment architectures require the following Splunk Enterprise components:
- Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.
- Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.
- Universal Forwarder: A streamlined Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.
- Deployment server (optional): A Splunk Enterprise instance that deploys apps and add-ons to indexers and forwarders. You can use the deployment server to install
Splunk_TA_stream
to new universal forwarders across a Stream distributed deployment.
For more information on Splunk Enterprise components, see Components of a Splunk Enterprise deployment
Single instance deployment
You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single instance deployment, which is ideal for a lab or test environment, can support one or two users running concurrent searches.
Distributed deployment
A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.
For a distributed deployment of Splunk App for Stream:
Search Heads
Install Splunk App for Stream (splunk_app_stream
) on one or more search heads.
Note: Splunk App for Stream does not currently support search head clustering.
Indexers
Install Splunk_TA_stream
on all indexers.
Forwarders
Install Splunk_TA_stream
on any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures."
Deployment server and Distributed Management Console (DMC)
In a small, single search head deployment, both the deployment server and Distributed Management Console (DMC) components can run on the same Splunk Enterprise instance on which you install splunk_app_stream
.
In deployments with large numbers of forwarders (over 100), we recommend that you run splunk_app_stream
on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning."
How it works
The streamfwd
binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The streamfwd
binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.
Use the Streams Config UI inside splunk_app_stream
to configure the specific wire data protocols (such as http, tcp, dns) that you want the streamfwd
binary to capture.
Use the streamfwd.xml
file in Splunk_TA_stream/local
to configure system-level parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for the streamfwd
binary. See "Configure Stream forwarder".
While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install splunk_app_stream
on more than one search head.
How streamfwd communicates with splunk_app_stream
The streamfwd
binary pings splunk_app_stream
at regular intervals over HTTP port 8000. If streamfwd
detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated streamfwd
configuration data.
The streamfwd
binary pings splunk_app_stream
at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of splunk_app_stream
is stored in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf
. See Specify the location of splunk_app_stream.
Note: streamfwd
pings splunk_app_stream
at default intervals of 5 seconds. To change the ping interval, use the <PingInterval>
element in streamfwd.xml
. See Configure Stream forwarder.
Deployment requirements | Network collection architectures |
This documentation applies to the following versions of Splunk Stream™: 6.2.0, 6.2.1, 6.2.2, 6.3.0, 6.3.1, 6.3.2
Feedback submitted, thanks!