Splunk® Enterprise

Distributed Deployment Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Reference hardware

At higher data indexing rates and/or user counts, you must take into account the differing needs of indexers and search heads. Dedicated search heads do not need extremely fast disk throughput, nor do they need much local storage. They do, however, require far more CPU resources than indexers do.

The reference indexer for a distributed Splunk Enterprise deployment is somewhat different from a reference indexer in a single-server deployment. Of particular note is the disk subsystem: a reference indexer in a distributed deployment has significantly larger disk throughput requirements over a single-server reference indexer. The main reason for this is that an indexer in a distributed deployment indexes more data and handles more search requests than one in a single-server deployment.

Following are the recommendations for both search heads and indexers in a distributed deployment:

Dedicated search head

  • Intel 64-bit chip architecture
  • 4 CPUs, 4 cores per CPU, at least 2 Ghz per core
  • 12 GB RAM
  • 2 x 300 GB, 10,000 RPM SAS hard disks, configured in RAID 1
  • Standard 1Gb Ethernet NIC, optional 2nd NIC for a management network
  • Standard 64-bit Linux or Windows distribution

Note: Since search heads require raw computing power and are likely to become CPU-bound, it is better to add additional CPU cores to a search head if faster performance per server is desired. If search workload dictates, a search head could require 4 CPUs with 6 cores per CPU. Regardless, the guideline of 1 core per active user still applies. Additionally:

  • Don't forget to account for scheduled searches in your CPU allowance. A typical search request requires 1 CPU core.
  • Depending on the type of search you use, you might need to add CPU cores to account for the increased load those search types cause.

Indexer

Indexers in a distributed deployment configuration have higher disk I/O bandwidth requirements than indexers in a non-distributed environment. This is because indexers must both write new data and service the remote requests of search heads.

  • Intel 64-bit chip architecture
  • 2 CPUs, 6 cores per CPU, at least 2 Ghz per core
  • 12 GB RAM
  • Disk subsystem capable of 1200 average input/output operations per second (random IOPS)
  • Standard 1Gb Ethernet NIC, optional 2nd NIC for a management network
  • Standard 64-bit Linux or Windows distribution

At higher daily volumes, local disk will likely not provide cost-effective storage for the time frames where speedy search is desired. In these cases, we suggest deploying fast attached storage or networked storage, such as storage area networks (SAN) over fiber. While there are too many types of storage to recommend, consider these guidelines when planning your storage infrastructure:

  • Indexers do many bulk reads.
  • Indexers do many disk seeks.

Therefore:

  • More disks (specifically, more spindles) are better for indexing performance.
  • Total throughput of the entire system is important, however:
  • The ratio of disks to disk controllers in a particular system should be higher, similar to how you configure a database server.

Ratio of indexers to search heads

Technically, there is no practical limitation on the number of search heads an indexer can support, or the number of indexers a search head can search against. However, systems limitations suggest a ratio of approximately 8 indexers to 1 search head in most use cases. That is a rough guideline - if you have many searchers compared to your total data volume, then more search heads will increase search efficiency. In some cases, the best use of a separate search head is to populate summary indexes. This search head then acts like an indexer to the primary search head that users log into.

PREVIOUS
How Splunk Enterprise looks through your data
  NEXT
Accommodate concurrent users and searches

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14


Comments

Hi Supersleepwalker,<br /><br />I've fixed that reference. A search head can always search multiple indexers, so the number of indexers will always be higher.

Malmoore
January 30, 2014

When you say "8 to 1" it's not clear which is search heads and which is indexers, since you discuss them in both orders in the first half of the sentence.

Supersleepwalker
January 30, 2014

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters