Use separate partitions for index data
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
Use separate partitions for index data
Splunk can use separate disks and partitions for its index data. It's possible to configure Splunk to use many disks/partitions/filesystems on the basis of indexes and warm/cold, so long as you mount them correctly and configure the DB rolling. However, we recommend that you use a single high performance file system to hold your Splunk index data for the best experience.
Splunk indexes roll through four stages:
- Hot - open for writing. There are multiple hot buckets. Searchable.
- Warm - data rolled from hot. There are many warm buckets. Searchable.
- Cold - data rolled from warm. There are many cold buckets. Searched only when the search specifies a time range included in these files.
- Frozen - buckets entering the frozen state are immediately deleted.
If you do use separate partitions, the most common way to arrange Splunk's index data is to keep the hot and warm buckets on the local machine, and to keep the cold bucket on a separate array or disks (for longer term storage). You want to run your hot and warm buckets on a machine with partitions that read and write fast (since you'll be doing a majority of your search operations on hot and warm). Cold should be on a reliable array of disks.
Bucket flow:
- Buckets roll from hot to warm when they reach the specified size (
maxDataSize) - Buckets roll from warm to cold when the number of warm buckets exceeds the configured maximum count (
maxWarmDBCount) - Buckets stay in cold (or warm) until they are selected for archiving
Set up separate partitions
Set up partitions just as you'd normally set them up in any operating system. Mount the disks/partitions, and make sure Splunk points to the correct path in indexes.conf.
First, add the correct paths in $SPLUNK_HOME/etc/system/local/indexes.conf. Set paths on a per-index basis -- under an [$INDEX] entry.
homePath = <path on server>
- The path that contains the hot and warm databases and fields for the index.
- Databases that are warm have a handle open to them at all times in splunkd.
- CAUTION: Path MUST be writable.
coldPath = <path on server>
- The path that contains the cold databases for the index.
- Cold databases are opened as needed when searching.
- CAUTION: Path MUST be writable.
thawedPath = <path on server>
- The path that contains the thawed (resurrected) databases for the index.
This documentation applies to the following versions of Splunk: 4.0 , 4.0.1 , 4.0.2 , 4.0.3 , 4.0.4 , 4.0.5 , 4.0.6 , 4.0.7 , 4.0.8 , 4.0.9 , 4.0.10 , 4.0.11 View the Article History for its revisions.