Splunk Stream

Installation and Configuration Manual

This documentation does not apply to the most recent version of Splunk Stream. For documentation on the most recent version, go to the latest release.

Deployment requirements

For a list of network data protocols that Splunk App for Stream supports for capture, see Supported Protocols.

Hardware requirements

Before you install Splunk App for Stream, make sure that your Splunk Enterprise deployment meets the requirements specified in "Introduction to capacity planning for Splunk Enterprise" in the Splunk Enterprise Capacity Planning Manual.

For details on Splunk component performance specifications and reference hardware requirements, see "Reference hardware" in the Splunk Enterprise Capacity Planning Manual.

For recommendations on scaling your Splunk Enterprise deployment for your specific performance requirements, see the "Performance questionnaire" in the Splunk Enterprise Capacity Planning Manual.

Note: Reference hardware recommendations refer only to the underlying Splunk Enterprise deployment on which your Splunk App for Stream installation runs. Depending on the throughput of the network data streams that you capture, additional processor, memory, and storage capacity might be required.

Hardware performance test summary

Based on our test data, App for Stream uses approximately 14% of a single CPU core for each 100 Mbps TCP traffic processed, when using the default streams configuration (only the udp and tcp flow data streams are enabled). This increases to approximately 21% when application layer protocol processing is enabled for unencrypted traffic (by enabling the http stream), and approximately 29% when the traffic is encrypted using SSL.

Test environment

The Splunk App for Stream test environment used:

  • A single VM with four virtual cores running CentOS 6 64-bit on Vmware ESX host, with App for Stream version 6.2.0 running on Splunk version 6.2.0.
  • Splunk_TA_stream (with streamfwd binary) installed on a universal forwarder, sending all data to a remote Splunk Indexer. ProcessingThreads is set to 4 in streamfwd.xml.
  • Neoload to simulate HTTP/S traffic.
  • 2 web servers running Nginx to respond to HTTP/S traffic with a static 100 KB HTML file.
  • Every HTTP request is sent using a unique TCP connection (worst case scenario).
  • Every HTTPS request is sent using a unique TCP connection and a unique SSL session (worst case scenario).
  • For the SSL scenario (HTTPS), the private RSA is configured such that all requests are being decrypted.

Test results

This table shows hardware performance at increasing bandwidths for non-SSL, SSL, and TCP streams.

Note: Hardware performance test results represent median values.

Test Data NON-SSL SSL TCP
Bandwidth (Mbps) 8 64 256 512 1000 8 64 256 512 8 64 256 512 1000
Events per second (eps) 19 154 646 1341 2677 19 151 637 1324 9 82 336 715 1482
CPU % of single core (streamfwd) 2 10 35 74 141 4 19 69 127 2 10 36 65 118
CPU % of single core (splunkd) 1 3 11 25 57 1 3 11 23 1 2 6 14 29
Memory MB (streamfwd) 409 408 376 413 652 389 382 363 405 329 334 345 384 654
Memory MB (splunkd) 108 108 108 108 109 112 112 107 109 106 106 106 107 110

Stream app test results 5.png

Supported operating systems

Splunk App for Stream 6.2.0 supports the following Linux, Mac, and Windows operating systems:

Linux

  • Linux kernel version 2.6.x or later (32- and 64-bit).
  • Red Hat Enterprise Linux 5 or later
  • CentOS 5 or later
  • Ubuntu 10 or later
  • Debian 5 or later

Linux kernel settings for high-volume packet capture

The default Linux kernel settings are not sufficient for high-volume packet capture. This can lead to missing packets and data loss. We recommend that you add the following kernel settings to your /etc/sysctl.conf file:

# increase kernel buffer sizes for reliable packet capture
net.core.rmem_default = 33554432
net.core.rmem_max = 33554432
net.core.netdev_max_backlog = 10000

Then run the following to reload the settings:

/sbin/sysctl -p

Mac OSX

  • Mac OSX version 10.8 or later.

Windows

  • Windows Server 2008R2 and later (64-bit).
  • Windows 7 (64-bit) Professional, Enterprise, and Ultimate editions.

Note: Splunk App for Stream supports running as Admin only on Windows.

Splunk Enterprise version

Splunk App for Stream is dependent on the Splunk Enterprise platform. Make sure to download and install the appropriate version of Splunk Enterprise before you install Splunk App for Stream.

Splunk App for Stream version 6.2.0 requires Splunk Enterprise version 6.1.x or later. (Splunk Enterprise version 6.2.1 or later recommended).

Splunk component requirements

Install the Splunk App for Stream on your Splunk forwarder, indexer, and search head, as applicable to your deployment. For details on Splunk App for Stream deployment architectures, see "Deployment architectures."

  • Forwarder: The forwarder sends network event data captured by the Stream Forwarder (streamfwd binary) to Splunk indexer(s) using the Wire Data modular input. Splunk App for Stream distributed deployments require universal forwarders.

By default, the universal forwarder sends a maximum of 256 Kbps of data to your Splunk indexers.  Depending on your traffic and configuration, streamfwd might generate more than this.  You can change or remove this threshold by modifying your $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf file to include:

[thruput]
maxKBps = 0

By default, the universal forwarder does not forward events to _internal index.  You must enable this for the dashboards in Splunk App for Stream to work. To enable, modify your $SPLUNK_HOME/etc/system/local/outputs.conf to include the following:

[tcpout]
forwardedindex.2.whitelist = (_audit|_introspection|_internal)
  • Indexer: Indexers receive and index network event data sent from Splunk forwarders.
  • Search head: The search head is where you perform search and analysis operations on your network event data. Search Heads provide search time knowledge for field extractions and event types.

For more information about Splunk components, see "Components of a Splunk Enterprise deployment" in the Splunk Enterprise Capacity Planning Manual.

Supported browsers

Splunk App for Stream 6.2.0 supports these browsers:

  • Chrome (latest)
  • Safari (latest)
  • Firefox (latest) (version 10.x is not supported)
  • Internet Explorer 9 or later. Internet Explorer version 9 is not supported in compatibility mode.

Splunk licensing

Splunk licenses are based on the amount of data stored by your Splunk indexers per day. For detailed information, see "How Splunk licensing works."

Last modified on 16 March, 2015
About Splunk App for Stream   Deployment architectures

This documentation applies to the following versions of Splunk Stream: 6.2.0, 6.2.1, 6.2.2


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters