Splunk data management

Splunk data management

Splunk stores all processed data in indexes. Splunk ships with preconfigured indexes in $SPLUNK_HOME/etc/system/default/indexes.conf. The following is a list of the indexes and what they contain:

• main: All processed data. Unless specified otherwise, this is the default index for all your data.
• history: All search history.
• splunklogger: Internal logs.
• summary: Summary indexing searches.
• _audit: Events from the file system change monitor and auditing.
• _blocksignature: Event block signatures.
• _internal: Metrics from Splunk's processors.
• _thefishbucket: Internal information on file processing.

Each index is a collection of databases located in $SPLUNK_HOME/var/lib/splunk. Databases are named as db_<starttime>_<endtime>_<seq_num>.

By default, Splunk searches through the main index. If you want to restrict your search to an index other than main, use index= to specify the index in your search. For example, to search for userid=henry.gale only in the hatch index:

Index management

You can add and remove indexes or move existing indexes.

Manage your indexes by configuring:

• Disk usage settings
• Data retirement policies
• How Splunk archives data

Configure Splunk to use multiple partitions for its datastore, or use a write once, read many storage device.

Configuration files for index management

Splunk's indexes are managed through indexes.conf. Edit this file in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work.

Note: Settings in indexes.conf are configured per index (rather than a global server setting).

Before making changes to how Splunk manages data consider:

• Your data retention policies.
• How much data your Splunk deployment will consume (for example: 50GB/day).
• Where your Splunk index datastores will live.

• Was this topic useful?
• Post a comment