Deployment architectures

This topic covers common deployment architectures for Splunk App for Stream.

Stream deployment architectures

Splunk App for Stream supports the following deployment architectures:

• Single instance deployment
• Distributed deployment

These deployment architectures require the following Splunk Enterprise components:

• Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.

• Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.

• Universal Forwarder: A streamlined Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.

• Deployment server (optional): A Splunk Enterprise instance that deploys apps and add-ons to indexers and forwarders. You can use the deployment server to install Splunk_TA_stream to new universal forwarders across a Stream distributed deployment.

For more information on Splunk Enterprise components, see Components of a Splunk Enterprise deployment

Single instance deployment

You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single instance deployment, which is ideal for a lab or test environment, can support one or two users running concurrent searches.

Distributed deployment

A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.

For a distributed deployment of Splunk App for Stream:

Search Heads

Install Splunk App for Stream (splunk_app_stream) on one or more search heads.

Note: Splunk App for Stream does not currently support search head clustering.

Indexers

Install Splunk_TA_stream on all indexers.

Forwarders

Install Splunk_TA_stream on any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures."

Deployment server and Distributed Management Console (DMC)

In a small, single search head deployment, both the deployment server and Distributed Management Console (DMC) components can run on the same Splunk Enterprise instance on which you install splunk_app_stream.

In deployments with large numbers of forwarders (over 100), we recommend that you run splunk_app_stream on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning."

How it works

The streamfwd binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The streamfwd binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.

Use the Streams Config UI inside splunk_app_stream to configure the specific wire data protocols (such as http, tcp, dns) that you want the streamfwd binary to capture.

Use the streamfwd.xml file in Splunk_TA_stream/local to configure system-level parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for the streamfwd binary. See "Configure Stream forwarder".

While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install splunk_app_stream on more than one search head.

How streamfwd communicates with splunk_app_stream

The streamfwd binary pings splunk_app_stream at regular intervals over HTTP port 8000. If streamfwd detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated streamfwd configuration data.

The streamfwd binary pings splunk_app_stream at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of splunk_app_stream is stored in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf. See Specify the location of splunk_app_stream.

Note: streamfwd pings splunk_app_stream at default intervals of 5 seconds. To change the ping interval, use the <PingInterval> element in streamfwd.xml. See Configure Stream forwarder.