Performance tuning Splunk

Performance tuning Splunk

By default, Splunk delivers high indexing throughput, fast search speeds, and dense storage. However, each system is unique, and you may find that tuning Splunk produces significant performance boosts. This section shows a summary of performance tuning recommendations to help boost Splunk's performance in your environment.

Hardware considerations

Splunk's performance is affected by the quality of hardware in the system. Provide the best performance possible for your Splunk Server by maximizing the quality of hardware you use. Different hardware components have different impacts on performance:

• Splunk can use up to 4 cores (not hyper-threaded) for indexing, and up to 4 more cores for each concurrent search.
• Run Splunk on an 8-core server for a significant search performance gain (+30-40%) when using multiple indexes.
• Run Splunk on a 64-bit platform to increase the scaling and speed of searching. Running on a 64-bit platform allows you to search 12x the amount of data (10GB buckets instead of 800MB in 32-bit) in equivalent time and memory as 32-bit platforms running Splunk.
• Use faster hard drives to improve search speeds. Fast SCSI drives with a quality RAID controller can increase indexing speed up to 1.6x, and search speed up to 10x during long-running, complex searches.
• Use a networking controller, or a dedicated TCP card to off-load networking operations from the CPU to improve searching and indexing speeds as well as network performance.
• Splunk can run on a virtual machine. Virtual machines allow Splunk to run in a dedicated environment that is not native to the system. However, virtual environments may degrade performance.

Hardware considerations grow more complex when working with Splunk distributed deployments.

Increase indexing performance

Improve indexing performance by tuning Splunk's time stamp extraction settings, segmentation, and other indexing properties. These settings are controlled in Splunk's various configuration files. Learn more about how to tune indexing here.

Increase search speed

Tuning your search speed also involves tuning settings in Splunk's configuration files. Segmentation, timestamping settings, and settings in Splunk Web affect your search speed. Learn more about how to tune your search speed here.

Improve storage efficiency

Splunk comes configured out-of-the-box, able to compress raw data by approximately 40-50%. In some cases, it is possible to tune Splunk's storage compression to 12% of raw data size. Tune Splunk's storage ratio by configuring segmentation settings within configuration files. In some cases, storage ratio is inversely proportional to search convenience. Learn how to configure your storage efficiency here.

Reduce the CPU and memory footprint

Searching massive amounts of data efficiently may require tuning Splunk's CPU and memory usage. Learn how to improve CPU and memory usage and increase overall throughput here.

Utilize multiple CPUs or cores

Increasing the number of CPUs and active cores in your system can improve indexing and search performance. Splunk uses cores for true index threading (not hyper-threading). We expect Splunk to perform better with more cores because the cache is shared; hence, it is closer if two threads use the same memory. Learn more about how to make use of a multi-CPU/core system here.

64-bit operating systems

64-bit platforms improve Splunk's ability to scale search and index operations. The increased memory results in an order of magnitude more of data that can be searched in the same amount of time and and memory as a 32-bit system. Learn how to tweak a 64-bit system here.

• Was this topic useful?
• Post a comment