Splunk® Enterprise

Distributed Deployment Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Distribute indexing and searching

This topic discusses the concepts and hardware requirements for distributing the indexing and searching components of your Splunk Enterprise deployment.

Concepts of distributed indexing and searching

You scale your Splunk Enterprise deployment by dedicating searching and indexing across multiple servers. Indexers bring in, store, and search the data. Search heads manage search requests and present results.

Since indexers require much more disk I/O throughput than search heads do, you give your environment more indexing capacity by reducing the overhead required for searching. The key points to remember are:

  • The more indexers you add to the deployment, the faster data is consumed and prepared for searches.
  • The more search heads you add to the deployment, the faster you are able to find the data you indexed.

Considerations for search performance vs. indexing performance

While the two points shown above are best practice for improving indexing speed, there are some important caveats to note as well, particularly when it comes to search speed.

As your indexers consume data, they store it in buckets - individual elements of an index. As more data comes in, the number of buckets increases. An increased number of buckets - particularly those which hold smaller amounts of data - can impact search speed because of the throughput required to navigate through those buckets for the data that you're searching.

Additionally, as the number of buckets increases, the indexer must manage the buckets. It does this by "rolling" buckets - thus making room for new incoming data. This procedure takes up I/O cycles as well - cycles that could be used to fetch events for search requests.

The key points to understand are:

  • You can't necessarily improve search performance simply by adding search heads to your distributed deployment. A mix of search heads and indexers is vital.
  • The number and types of search also impact indexer performance. Some search types tax an indexer's CPU, others apply pressure to the disk subsystem.

More detail about how to plan for simultaneous searches is found in "Accommodate concurrent users and searches" in this manual.]

PREVIOUS
Hardware capacity planning for a distributed Splunk Enterprise deployment
  NEXT
How Splunk Enterprise looks through your data

This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters