Hardware capacity planning for a distributed Splunk Enterprise deployment
If you have larger indexing or searching requirements, run Splunk Enterprise apps or solutions that generate or execute a lot of saved searches, or regularly employ I/O-intensive searches, then you should scale your deployment to address the increased resource overhead that those operations incur. For an overview of what a distributed Splunk Enterprise deployment is, review "Distributed Splunk Enterprise overview" in this manual.
In many cases, this involves using distributed search to run searches in parallel across multiple indexers at once. You can gather data from machines using Splunk Enterprise forwarders and, optionally, configure those servers to send data to multiple indexers at once to reduce search time. For information on the individual elements of a Splunk Enterprise deployment, read "Components of a Splunk Enterprise deployment" in the Installation Manual.
Estimate hardware requirements
While determining the hardware requirements for your distributed Splunk Enterprise deployment, there are a number of things you must consider.
You must understand how various Splunk Enterprise activities affect the resource overhead required to perform them. Each of the following activities has a direct impact on the overall performance of Splunk Enterprise:
- The amount of data you index.
- The number of concurrent users.
- The number of saved searches you run.
- The types of search you employ.
- The number of apps or solutions you implement.
- When you run apps, whether or not those apps execute a large number of saved searches.
When you add more indexers to a deployment, you increase the amount of available indexing capacity by reducing the indexing overhead per server. Consequently, reduced indexing overhead also means reduced search time.
But that is only half the story. While Splunk Enterprise scales across multiple indexers, the amount of indexing throughput becomes less important as either the number of concurrent users or saved searches increases. Additionally, depending on the kinds of searches you employ against your data, the resource needs for searching can become as important as the resource needs for indexing.
For additional information on estimating your hardware requirements, read the following topics, all in this manual:
- "Distribute indexing and searching" - for details on how to begin structuring your distributed environment.
- "How Splunk Enterprise looks through your data" - to learn what the different search types are, and how they impact performance on an indexer.
- "Reference hardware" - to learn about the reference servers and the indexing and searching performance they are capable of.
- "Accommodate concurrent users and searches" - for various scenarios on addressing search performance.
- "How Splunk apps affect resource requirements" - for additional information on how Splunk apps consume computing resources.
Considerations for clusters
There are some additional hardware issues to consider if you're implementing Splunk Enterprise clusters. See "System requirements and other deployment considerations" in the Managing Indexers and Clusters manual.
How deployments scale
Distribute indexing and searching
This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14