Performance tuning Splunk

Performance tuning Splunk

Splunk comes out-of-the-box with the ability to deliver higher indexing throughput, faster search speeds, and denser storage than any of its competitors. Tuning Splunk's indexing, searching, and storage requirements can produce significant performance boosts to its baseline performance specifications. This section will show you a summary of performance tuning recommendations that will help you unlock Splunk's performance potential.

Hardware considerations

Splunk's performance is affected by the quality of hardware in the system. Provide the best performance possible for your Splunk server by maximizing the quality of hardware you use. Different hardware components have different impacts on performance:

• Use faster CPUs with more cores. More and faster CPUs speed up search and indexing. Splunk can use up to 4 cores (not hyper-threaded) for indexing, and up to 4 more cores for each concurrent search.
• Run Splunk on an 8-core server for a significant search performance gain (+30-40%) when using multiple indexes.
• Run Splunk on a 64-bit platform to increase the scaling and speed of searching. Running on a 64-bit platform will allow you to search 12x the amount of data (10GB buckets instead of 800MB in 32-bit) in equivalent time and memory as 32-bit platforms running Splunk.
• Use faster hard drives to improve search speeds. Fast SCSI drives with a quality RAID controller can increase indexing speed up to 1.6x, and search speed up to 10x during long-running, complex searches.
• Use a networking controller, or a dedicated TCP card to off-load networking operations from the CPU to improve searching and indexing speeds as well as network performance.

Hardware considerations grow more complex when working with Splunk distributed search deployments.

Increase indexing performance

Indexing performance can be improved by tuning Splunk's timestamp extraction settings, segmentation of events, and advanced features (such as event type discoverer and automatic event typing). These settings are controlled in Splunk's various configuration files. Learn more about how to tune your indexing here.

Increase search speed

Tuning your search speed also involves tuning settings in Splunk's configuration files. Segmentation, timestamping settings, and Splunk's advanced features affect your search speed. Learn more about how to tune your search speed here.

Improve storage efficiency

Splunk comes configured out-of-the-box, able to compress raw data by approximately 40-50%. In some cases, it is possible to tune Splunk's storage compression to 12% of raw data size. Splunk's storage ratio is tuned by configuring your segmentation settings within configuration files. Often, storage ratio is inversely proportional to search convenience provided by some of Splunk's advanced features. Learn how to configure your storage efficiency here.

Reduce the CPU and memory footprint

Searching massive amounts of data efficiently may require tuning Splunk's CPU and memory usage. Learn how to improve CPU and memory usage and increase overall throughput here.

Utilize multiple CPUs

Increasing the number of CPUs and active cores in your system will improve indexing and search performance. Splunk uses cores for true index threading (not hyper-threading). Learn more about how to make use of a multi-CPU/core system here.

64-bit operating systems

64-bit platforms improve Splunk's ability to scale search and index operations. The increased memory width results in an order of magnitude more of data that can be searched in the same amount of time and and memory as a 32-bit system. Learn how to tweak a 64-bit system here.

Virtual machines

Splunk can be run on a virtual machine. Virtual machines allow for Splunk to run in a chosen environment that is not native to the system you are running. Virtual environments degrade performance. Learn more about how to optimize your virtual environment for Splunk here.

• Was this topic useful?
• Post a comment