Splunk Stream

Installation and Configuration Manual

This documentation does not apply to the most recent version of Splunk Stream. For documentation on the most recent version, go to the latest release.

Deployment requirements

For a list of network data protocols that Splunk App for Stream supports for capture, see Supported Protocols.

Hardware requirements

Before you install Splunk App for Stream, make sure that your Splunk Enterprise deployment meets the requirements specified in "Introduction to capacity planning for Splunk Enterprise" in the Splunk Enterprise Capacity Planning Manual.

For details on Splunk component performance specifications and reference hardware requirements, see "Reference hardware" in the Splunk Enterprise Capacity Planning Manual.

For recommendations on scaling your Splunk Enterprise deployment for your specific performance requirements, see the "Performance questionnaire" in the Splunk Enterprise Capacity Planning Manual.

Note: Reference hardware recommendations refer only to the underlying Splunk Enterprise deployment on which your Splunk App for Stream installation runs. Depending on the throughput of the network data streams that you capture, additional processor, memory, and storage capacity might be required.

Hardware performance test summary

Based on our test data, each 100 events per second (eps) (approx. 32Mbps) increase in non-SSL traffic requires an additional 6-8% CPU load capacity. Each 100 eps increase in SSL traffic requires an additional 13-18% CPU load capacity.

Test environment

The Splunk App for Stream test environment used:

  • A single VM running CentOS 6 64-bit on Vmware ESX host, with App for Stream version 6.0 running on Splunk version 6.1.
  • Splunk_TA_stream (with streamfwd binary) installed on a universal forwarder, sending all data to a remote Splunk Indexer. App for Stream using default configuration with UDP, TCP and HTTP streams enabled.
  • Jmeter to simulate HTTP traffic and Siege to generate HTTPS traffic, with a new connection for each request (no Keep-Alive).
  • Nginx to respond to HTTP/S traffic with a static 100 KB HTML file.

Test results

This table shows hardware performance at increasing bandwidths for non-SSL and SSL streams.

Note: Hardware performance test results represent median values.

Test Data NON-SSL SSL
Bandwidth (Mbps) 8 64 256 512 8 64 256 512
Events per second (eps) 24 169 650 1370 24 170 676 1,318
CPU % of single core (streamfwd) 2 10 32 57 3 22 69 93
CPU % of single core (splunkd) 0 4 14 29 1 5 15 21
Memory MB (streamfwd) 128 179 201 229 132 156 196 258
Memory MB (splunkd) 125 136 136 136 118 122 123 123

StreamApp test results 4.png


This table shows extrapolated CPU load at increasing bandwidths for non-SSL and SSL streams:

Bandwidth (Mbps) 8 Mbps 64 Mbps 256 Mbps 512 Mbps 1 Gbps* 2 Gbps* 5 Gbps* 10 Gbps*
Events per second (eps) 24 169 650 1,370 2,740 5,480 13,700 27,400
Combined CPU (NON-SSL) 2 14 46 86 172 344 860 1,720
Combined CPU (SSL) 4 27 84 114 228 456 1,140 2,280
CPU per 100 eps (NON-SSL) 8 8 7 6
CPU per 100 eps (SSL) 17 16 13 18
*Extrapolation from test data.

StreamApp test results 5.png

Supported operating systems

Splunk App for Stream 6.1.x supports these operating systems:

Linux

  • Linux kernel version 2.6.x or later (32- and 64-bit).
  • Red Hat Enterprise Linux 5 or later
  • CentOS 5 or later
  • Ubuntu 10 or later
  • Debian 5 or later

See supported protocols for Linux

Linux kernel settings for high-volume packet capture

The default Linux kernel settings are not sufficient for high-volume packet capture. This can lead to missing packets and data loss. We recommend that you add the following kernel settings to your /etc/sysctl.conf file:

# increase kernel buffer sizes for reliable packet capture
net.core.rmem_default = 33554432
net.core.rmem_max = 33554432
net.core.netdev_max_backlog = 10000

Then run the following to reload the settings:

/sbin/sysctl -p

Mac OSX

  • Mac OSX version 10.8 or later.

See supported protocols for Mac OSX.

Splunk Enterprise version

Splunk App for Stream is dependent on the Splunk Enterprise platform. Make sure to download and install the appropriate version of Splunk Enterprise before you install Splunk App for Stream.

Splunk App for Stream version 6.1.x requires Splunk Enterprise version 6.1.x or later. (Splunk Enterprise version 6.2.1 recommended).

Splunk component requirements

Install the Splunk App for Stream on your Splunk forwarder, indexer, and search head, as applicable to your deployment. For details on Splunk App for Stream deployment architectures, see "Deployment architectures."

  • Forwarder: The forwarder sends network event data captured by the Stream Forwarder (streamfwd binary) to Splunk indexer(s) using the Wire Data modular input. Splunk App for Stream distributed deployments require universal forwarders.

By default, the universal forwarder sends a maximum of 256 Kbps of data to your Splunk indexers.  Depending on your traffic and configuration, streamfwd might generate more than this.  You can change or remove this threshold by modifying your $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf file to include:

[thruput]
maxKBps = 0

By default, the universal forwarder does not forward events to _internal index.  You must enable this for the dashboards in Splunk App for Stream to work. To enable, modify your $SPLUNK_HOME/etc/system/local/outputs.conf to include the following:

[tcpout]
forwardedindex.2.whitelist = (_audit|_introspection|_internal)
  • Indexer: Indexers receive and index network event data sent from Splunk forwarders.
  • Search head: The search head is where you perform search and analysis operations on your network event data. Search Heads provide search time knowledge for field extractions and event types.

For more information about Splunk components, see "Components of a Splunk Enterprise deployment" in the Splunk Enterprise Capacity Planning Manual.

Supported browsers

Splunk App for Stream 6.1.0 supports these browsers:

  • Chrome (latest)
  • Safari (latest)
  • Firefox (latest) (version 10.x is not supported)
  • Internet Explorer 9 or later. Internet Explorer version 9 is not supported in compatibility mode.

Splunk licensing

Splunk licenses are based on the amount of data stored by your Splunk indexers per day. For detailed information, see "How Splunk licensing works."

Last modified on 10 February, 2015
About Splunk App for Stream   Deployment architectures

This documentation applies to the following versions of Splunk Stream: 6.1.0, 6.1.1


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters