Performance expectations for the Splunk App for CEF
This page provides reference information about the performance testing of the Splunk App for CEF version 2.0.0 performed by Splunk, Inc. Use this information to size your environment based on your data volume expectations and the existing resource usage of the indexers.
Many factors affect performance results, including file size, file compression, event size, custom datamodel definitions, number of concurrent searches, indexed realtime settings, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.
Splunk ran performance tests on one, three, and five indexers without indexed real-time searches enabled. The hardware specifications for the indexers are as follows:
- CentOS 6, 64-bit
- 12 hyperthreaded Intel(R) Xeon(R) CPU X5650 @ 2.67GHz, 24 cores total
- 96 GB RAM
The following graph and table display the performance test results for four different data volumes using a single search and an average event size of 710 bytes.
Many production scenarios for the Splunk App for CEF require several concurrent searches, each based on a different data model. See How concurrent users and searches impact performance in the Splunk Enterprise Capacity Planning Manual for information to help you to size your environment to accommodate multiple searches.
|1 indexer||3 indexers||5 indexers|
|1 million events||130.4 seconds
about 5.19 MB/s
about 438 GB/day
about 9.54 MB/s
about 813 GB/day
about 19.75 MB/s
about 1666 GB/day
|3.7 million events||479.9 seconds
about 5.21 MB/s
about 440 GB/day
about 10.09 MB/s
about 851 GB/day
about 22.29 MB/s
about 1881 GB/day
|6.9 million events||915.6 seconds
about 5.1 MB/s
about 430 GB/day
about 10.26 MB/s
about 866 GB/day
about 22.61 MB/s
about 1908 GB/day
|8 million events||1056.3 seconds
about 5.13 MB/s
about 433 GB/day
about 10.37 MB/s
about 873 GB/day
about 22.87 MB/s
about 1930 GB/day
Version 1.0.0 to 2.0.0 comparison
The 2.0.0 version rearchitects the data output functionality of the app by moving the output jobs to the indexers for improved throughput and scalability. Due to the deployment differences between 1.0.0 and 2.0.0, direct comparisons on identical environments are not possible.
Splunk ran baseline comparative tests on similar hardware using a Splunk App for CEF 1.0.0 deployment with a single search head, testing the same data volumes as for the 2.0.0 testing. Throughput with the 1.0.0 app ranged between 3.4 MB/s for 1 million events to 3.9 MB/s for 8 million events. When compared to the 2.0.0 performance test results, throughput performance increased with the 2.0.0 app by 30-70% using just a single indexer, and by 480-620% using five indexers.
Common Event Format conversion searches can cause resource contention with other searches running on the same instance. To maximize your performance, consider taking the following actions:
- Enable indexed real-time searching for the individual CEF searches to reduce load on the indexers.
- Reduce or eliminate the number of other searches running concurrently on the indexers that run the CEF conversion searches.
- Add indexers to horizontally scale your system.
- Build your indexers according to the Splunk Enterprise reference hardware standard.
- Use the fastest available disk subsystems, such as SSD, for your search heads and indexers.
Hardware and software requirements for the Splunk App for CEF
Install the Splunk App for CEF
This documentation applies to the following versions of Splunk® App for CEF: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0