Splunk® Enterprise

Managing Indexers and Clusters of Indexers

Download manual as PDF

Splunk Enterprise version 5.0 reached its End of Life on December 1, 2017. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Configure index storage

You configure indexes in indexes.conf. How you edit indexes.conf depends on whether you're using index replication, also known as "clustered indexing":

  • For non-clustered indexes, edit a copy of indexes.conf in $SPLUNK_HOME/etc/system/local/ or in a custom app directory in $SPLUNK_HOME/etc/apps/. Do not edit the copy in $SPLUNK_HOME/etc/system/default. For information on configuration files and directory locations, see "About configuration files".
  • For clustered indexes, edit a copy of indexes.conf on the cluster master node and then distribute it to all the peer nodes, as described in "Configure the peer indexes".

This table lists the key indexes.conf attributes affecting buckets and what they configure. It also provides links to other topics that show how to use these attributes. For the most detailed information on these attributes, as well as others, always refer to the indexes.conf spec file.

Attribute What it configures Default For more information, see ...
homePath The path that contains the hot and warm buckets. (Required.)

This location must be writable.

$SPLUNK_HOME/var/lib/splunk/ defaultdb/db/ (for the default index only) Use multiple partitions for index data
coldPath The path that contains the cold buckets. (Required.)

In clusters, this is also the location for all replicated copies of buckets. This location must be writable.

$SPLUNK_HOME/var/lib/splunk/ defaultdb/colddb/ (for the default index only) Use multiple partitions for index data
thawedPath The path that contains any thawed buckets. (Required.)

This location must be writable.

$SPLUNK_HOME/var/lib/splunk/ defaultdb/thaweddb/ (for the default index only) Use multiple partitions for index data
repFactor Determines whether the index gets replicated to other cluster peers. (Required for indexes on cluster peer nodes.) 0 (which means that the index will not get replicated to other peers; the correct behavior for non-clustered indexes). For clustered indexes, you must set repFactor to auto, which causes the index to get replicated. Configure the peer indexes
maxHotBuckets The maximum number of hot buckets. This value should be at least 2, to deal with any archival data. The main default index, for example, has this value set to 10. 3, for new, custom indexes. How Splunk ages data
maxDataSize Determines rolling behavior, hot to warm. The maximum size for a hot bucket. When a hot bucket reaches this size, it rolls to warm. This attribute also determines the approximate size for all buckets. Depends; see indexes.conf. Use multiple partitions for index data

Set a retirement and archiving policy

maxWarmDBCount Determines rolling behavior, warm to cold. The maximum number of warm buckets. When the maximum is reached, warm buckets begin rolling to cold. 300 Use multiple partitions for index data
maxTotalDataSizeMB Determines rolling behavior, cold to frozen. The maximum size of an index. When this limit is reached, cold buckets begin rolling to frozen. 500000 (MB) Set a retirement and archiving policy
frozenTimePeriodInSecs Determines rolling behavior, cold to frozen. Maximum age for a bucket, after which it rolls to frozen. 188697600 (in seconds; approx. 6 years) Set a retirement and archiving policy
coldToFrozenDir Location for archived data. Determines behavior when a bucket rolls from cold to frozen. If set, Splunk will archive frozen buckets into this directory just before deleting them from the index. If you don't set either this attribute or coldToFrozenScript, Splunk will just log the bucket's directory name and then delete it once it rolls to frozen. Archive indexed data
coldToFrozenScript Script to run just before a cold bucket rolls to frozen. If you set both this attribute and coldToFrozenDir, Splunk will use coldToFrozenDir and ignore this attribute. If you don't set either this attribute or coldToFrozenDir, Splunk will just log the bucket's directory name and then delete it once it rolls to frozen. Archive indexed data
homePath.maxDataSizeMB

coldPath.maxDataSizeMB

Maximum size for homePath (hot/warm bucket storage) or coldPath (cold bucket storage). If either attribute is missing or set to 0, its path is not individually constrained in size. None Configure index size according to bucket type
maxVolumeDataSizeMB Maximum size for a volume. If the attribute is missing, the individual volume is not constrained in size. None Configure index size with volumes

Note: For non-clustered indexes only, you can use Splunk Manager to configure the path to your indexes. Go to Splunk Manager > System settings > General settings. Under the section Index settings, set the field Path to indexes. After doing this, you must restart Splunk from the CLI, not from within Manager.

PREVIOUS
How Splunk stores indexes
  NEXT
Move the index database

This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters