Use multiple partitions for index data
Splunk can use multiple disks and partitions for its index data. It's possible to configure Splunk to use many disks/partitions/filesystems on the basis of multiple indexes and bucket types, so long as you mount them correctly and point to them properly from
indexes.conf. However, we recommend that you use a single high performance file system to hold your Splunk index data for the best experience.
If you do use multiple partitions, the most common way to arrange Splunk's index data is to keep the hot/warm buckets on the local machine, and to put the cold bucket on a separate array of disks (for longer term storage). You'll want to run your hot/warm buckets on a machine with with fast read/write partitions, since most searching will happen there. Cold buckets should be located on a reliable array of disks.
Important: Requirements for cold storage are entirely different if you're using clusters, as described below in "Clusters and the coldPath".
Configure multiple partitions
To configure multiple partitions:
1. Set up partitions just as you'd normally set them up in any operating system.
2. Mount the disks/partitions.
3. Edit indexes.conf to point to the correct paths for the partitions. You set paths on a per-index basis, so you can also set separate partitions for different indexes. Each index has its own
[<index>] stanza, where
<index> is the name of the index. These are the settable path attributes:
homePath = <path on server>
- This is the path that contains the hot and warm databases for the index.
- Caution: The path must be writable.
coldPath = <path on server>
- This is the path that contains the cold databases for the index.
- Caution: The path must be writable.
- Important: In a cluster, this path also serves as the location of all replicated copies of buckets - hot, warm, and cold. (The original copies of cluster buckets, however, reside in their normal locations, according to the type of bucket.) Therefore, the type of storage you use for the
coldPathdirectory has entirely different requirements with clusters, as described below, in "Clusters and the coldPath".
thawedPath = <path on server>
- This is the path that contains any thawed databases for the index.
Clusters and the coldPath
In a cluster, the storage used for the
coldPath directory should have the same characteristics as that used for
homePath storage. This is because all replicated copies of buckets reside in the
coldPath directory. It doesn't matter whether they're hot, warm, or cold. If you use slower storage for the
coldPath location, it will slow the overall performance of your cluster.
Unlike non-clustered indexers, where
coldPath typically contains infrequently accessed data and can therefore be located on slower disk arrays, clusters require strongly performing storage for the
coldPath location, to handle the needs of cluster operations. For example, some of the buckets in the
coldPath location on a cluster peer will be replicated hot bucket copies still being written to. Other buckets will be replicated warm copies, and the search head might be accessing them frequently. In addition, depending on how the cluster is configured and what occurs subsequently (in terms of peers going offline, etc.), the peer might need to convert bucket copies from non-searchable to searchable, entailing a considerable amount of processing on the
For more information on cluster operations, read "About clusters and index replication" and the topics that follow it. In particular, the topic "System requirements" has detailed information about cluster storage hardware.
Move the index database
Configure maximum index size
This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18