Splunk® Enterprise

Capacity Planning Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Distribute indexing and searching

This topic discusses the concepts and hardware requirements for distributing the indexing and searching components of your Splunk Enterprise deployment.

Concepts of distributed indexing and searching

Scale your Splunk Enterprise deployment by distributing searching and indexing across multiple machines. Indexers bring in, store, and search the data. Search heads manage search requests and present results.

Because indexers require much more disk I/O throughput than search heads do, give your environment more indexing capacity by reducing the overhead required for searching. The key points are as follows:

  • When you add indexers to the deployment, the aggregate rate of data consumption and total available storage increases, which provides additional capacity to manage increased search loads.
  • When you add search heads, you increase the number of concurrent users that the deployment can handle, thus providing capacity for increased search load.

The more search heads you add to the deployment, the faster you can find the data that you indexed.

Considerations for search performance versus indexing performance

While the two points in the previous section are best practice for improving indexing speed, some caveats apply, particularly when it comes to search speed.

As your indexers consume data, they store it in buckets, which are the individual elements of an index. As more data comes in, the number of buckets increases. A higher number of buckets can impact search speed because of the throughput required to navigate through those buckets for the data you want. This impact is noticeable for index buckets that hold smaller amounts of data.

As the number of buckets increases, the indexer must manage the buckets, which it does by "rolling" them to make room for new incoming data. This procedure takes up I/O cycles as well, which reduces the number of I/O cycles available to fetch events for search requests.

The key points to remember are:

  • Adding search heads to your distributed deployment does not guarantee improved search performance. A mix of search heads and indexers is vital for performance increases.
  • The number and types of search also impact indexer performance. Some search types tax an indexer's CPU, while others apply pressure to the disk subsystem.

See "Accommodate concurrent users and searches" in this manual for details on simultaneous searches.

Estimate your storage requirements
Accommodate many simultaneous searches

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters