Splunk® App for CEF

Deploy and Use Splunk App for CEF

Download manual as PDF

Download topic as PDF

Performance expectations for the Splunk App for CEF

This page provides reference information about the performance testing of the Splunk App for CEF version 2.0.0 performed by Splunk, Inc. Use this information to size your environment based on your data volume expectations and the existing resource usage of the indexers.

Many factors affect performance results, including file size, file compression, event size, custom datamodel definitions, number of concurrent searches, indexed realtime settings, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Testing architecture

Splunk ran performance tests on one, three, and five indexers without indexed real-time searches enabled. The hardware specifications for the indexers are as follows:

  • CentOS 6, 64-bit
  • 12 hyperthreaded Intel(R) Xeon(R) CPU X5650 @ 2.67GHz, 24 cores total
  • 96 GB RAM

Search performance

The following graph and table display the performance test results for four different data volumes using a single search and an average event size of 710 bytes.

Many production scenarios for the Splunk App for CEF require several concurrent searches, each based on a different data model. See How concurrent users and searches impact performance in the Splunk Enterprise Capacity Planning Manual for information to help you to size your environment to accommodate multiple searches.

Cef-200-performance-compressor.png

1 indexer 3 indexers 5 indexers
1 million events 130.4 seconds
7,669 events/s
about 5.19 MB/s
about 438 GB/day
70.3s
14,224 events/s
about 9.54 MB/s
about 813 GB/day
34.3s
29,155 events/s
about 19.75 MB/s
about 1666 GB/day
3.7 million events 479.9 seconds
7,710 events/s
about 5.21 MB/s
about 440 GB/day
248.5 seconds
14,889 events/s
about 10.09 MB/s
about 851 GB/day
112.4 seconds
32,918 events/s
about 22.29 MB/s
about 1881 GB/day
6.9 million events 915.6 seconds
7,536 events/s
about 5.1 MB/s
about 430 GB/day
455.3 seconds
15,154 events/s
about 10.26 MB/s
about 866 GB/day
206.6 seconds
33,397 events/s
about 22.61 MB/s
about 1908 GB/day
8 million events 1056.3 seconds
7,574 events/s
about 5.13 MB/s
about 433 GB/day
523.6 seconds
15,279 events/s
about 10.37 MB/s
about 873 GB/day
236.8 seconds
33,783 events/s
about 22.87 MB/s
about 1930 GB/day

Version 1.0.0 to 2.0.0 comparison

The 2.0.0 version rearchitects the data output functionality of the app by moving the output jobs to the indexers for improved throughput and scalability. Due to the deployment differences between 1.0.0 and 2.0.0, direct comparisons on identical environments are not possible.

Splunk ran baseline comparative tests on similar hardware using a Splunk App for CEF 1.0.0 deployment with a single search head, testing the same data volumes as for the 2.0.0 testing. Throughput with the 1.0.0 app ranged between 3.4 MB/s for 1 million events to 3.9 MB/s for 8 million events. When compared to the 2.0.0 performance test results, throughput performance increased with the 2.0.0 app by 30-70% using just a single indexer, and by 480-620% using five indexers.

Improve performance

Common Event Format conversion searches can cause resource contention with other searches running on the same instance. To maximize your performance, consider taking the following actions:

  • Enable indexed real-time searching for the individual CEF searches to reduce load on the indexers.
  • Reduce or eliminate the number of other searches running concurrently on the indexers that run the CEF conversion searches.
  • Add indexers to horizontally scale your system.
  • Build your indexers according to the Splunk Enterprise reference hardware standard.
  • Use the fastest available disk subsystems, such as SSD, for your search heads and indexers.
Last modified on 02 February, 2017
PREVIOUS
Hardware and software requirements for the Splunk App for CEF
  NEXT
Install the Splunk App for CEF

This documentation applies to the following versions of Splunk® App for CEF: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters