Deployment architectures

This topic covers common deployment architectures for Splunk App for Stream.

Splunk App for Stream components

Splunk App for Stream includes the following components:

• Splunk Stream Add-on (Splunk_TA_stream): Splunk_TA_stream captures network event data, and sends that data to Splunk indexers. Splunk_TA_stream contains the streamfwd binary (also called Stream Forwarder), which passively captures event data from network devices, and runs a Wire Data modular input that sends event data to STDOUT using XML streaming mode.

• Splunk App for Stream (splunk_app_stream): splunk_app_stream provides configuration management of network protocol events that the streamfwd binary captures. See Supported Protocols. splunk_app_stream also provides pre-built dashboards that display network interface and streamfwd binary process metrics.

Common deployment architectures

The most common Splunk App for Stream deployment architectures include:

• Single instance deployment
• Distributed deployment

These deployment architectures require the following Splunk Enterprise components:

• Search head: A Splunk Enterprise instance that is the central location for Splunk apps and search knowledge, hosting the users, and providing authentication and authorization. The search head also manages and directs search requests to indexers.

• Indexer: A Splunk Enterprise instance that processes search requests from search heads. The indexer also accepts incoming data streams from forwarders, transforms them into events, and writes the results into indexes.

• Forwarder: A Splunk Enterprise instance that collects and forwards data to the indexers. Forwarders are designed to load balance the data streams between indexers. Distributed deployments of Splunk App for Stream require the Splunk universal forwarder.

Single instance deployment

You can install Splunk App for Stream on a single Splunk Enterprise instance. A single instance serves as both a search head and indexer, accepting direct data streams along with storing and searching the data. A single server configuration, which is ideal for a lab or test environment, can support one or two users running concurrent searches.

Distributed deployment

A Splunk App for Stream distributed deployment can capture network event data from multiple network devices, including switches, routers, NICs, and so on. A distributed deployment architecture can apply to many types of medium and large enterprise network infrastructures.

For a distributed deployment of Splunk App for Stream:

• Install Splunk App for Stream (splunk_app_stream) on a Splunk Enterprise instance (6.0 or later).

• Install Splunk_TA_stream on any number of universal forwarders at the location(s) where you want to capture network data. See "Network collection architectures".

In a small, single search head deployment, both the Distributed Management Console (DMC) and deployment server components can run on the same Splunk Enterprise instance on which you install splunk_app_stream.

In deployments with large numbers of forwarders (over 100), we recommend that you run splunk_app_stream on the DMC host, and run the deployment server on a separate host. For more information on deployment server architecture and scalability, see "Deployment server prosivioning".

How it works

The streamfwd binary captures network event data from individual machines (such as each node of a subnet environment) or from a network SPAN or TAP. See Network collection architectures. The streamfwd binary sends captured network event data from universal forwarders to indexers over the Splunk App for Stream "Wire Data" modular input.

There are two types of configuration that apply to Splunk App for Stream:

• Streams Config UI: Use the Streams Config UI in splunk_app_stream (on the search head) to configure the specific wire data protocols (such as http, tcp, dns) that you want the streamfwd binary to capture.

• streamfwd.xml: Use the streamfwd.xml file in Splunk_TA_stream/local (on the universal forwarder) to configure parameters (specify IP address/ports, add network interfaces, enable SSL, and so on) for the streamfwd binary. See "Configure Stream forwarder".

While you can use multiple search heads to search and analyze captured network event data on your indexers, you do not need to install splunk_app_stream on more than one search head.

About Search Head Clustering

Splunk App for Stream is not compatible with search head clustering. As a workaround, you can choose one node of a search head cluster as the Stream configuration and management server. Once you install and configure Splunk App for Stream on the single search cluster node, all Stream Add-on forwarders must point to that search cluster node. You must also perform all stream capture configuration on the search cluster node that hosts the Splunk App for Stream.

How streamfwd communicates with splunk_app_stream

The streamfwd binary pings splunk_app_stream at regular intervals over HTTP port 8000. If streamfwd detects a change in the Streams UI configuration, it sends a GET request to the /streams REST API endpoint to retrieve the updated streamfwd configuration data.

The streamfwd binary pings splunk_app_stream at the location (URI) that you specify when you create a Wire Data modular input in Splunk Enterprise. The location of splunk_app_stream is stored in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf. See Specify the location of splunk_app_stream.

Note: streamfwd pings splunk_app_stream at default intervals of 5 seconds. To change the ping interval, use the <PingInterval> element in streamfwd.xml. See Configure Stream forwarder.