How Index Management Works
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
How Index Management Works
Splunk stores all processed data in indexes. Indexes, in turn, are stored in databases, which are located in $SPLUNK_HOME/var/lib/splunk. A database is a directory named db_<starttime>_<endtime>_<seq_num>. An index is a collection of database directories.
Splunk comes with preconfigured indexes:
- main: the default Splunk index. All processed data is stored here unless otherwise specified.
- splunklogger: Splunk keeps track of its internal logs in this index.
- history: all search history is stored here.
- metaevents: search here for any metaevents.
- _internal: this index includes metrics from Splunk's processors.
- sampledata: a small amount of sample data is stored here for training purposes.
- _thefishbucket: internal information on file processing.
You can add and remove indexes or move existing indexes.
Indexes can be searched via SplunkWeb. SplunkWeb searches automatically look through the default index (by default, main) unless otherwise specified. If you have created a new index, or want to search in any index that is not default, you must specify the index in your search:
index=hatch userid=henry.galeThis searches in the hatch index for the userid=henry.gale.
Data management
Index management is the main method for data management, including:
You can also set up Splunk to use multiple partitions for its datastore, or use a write once, read many storage device.
Configuration files for index management
Splunk's indexes are managed through the indexes.conf configuration file. You should make changes to this file in $SPLUNK_HOME/etc/bundles/local or create a new bundle.
Please note: settings in indexes.conf are per index rather than a global server setting.
Before making changes to how Splunk manages data consider:
- your company's data retention policies.
- how much data your Splunk deployment will consume (for example: daily rate (50GB/day)).
- where your Splunk index datastores will live.
This documentation applies to the following versions of Splunk: 3.0.1 , 3.0.2 , 3.1 , 3.1.1 , 3.1.2 , 3.1.3 , 3.1.4 View the Article History for its revisions.