Disk usage
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
Disk usage
There are several methods for controlling disk space used by Splunk. Most disk space will be used by Splunk's indexes and compressed log files (collectively called the database). If you run out of disk space, Splunk will stop indexing. You can set a minimum free space limit to control how low you will let free disk space fall before indexing stops. Indexing will resume once you space exceeds the minimum.
Set minimum free disk space
Use settings in Splunk Web to set a minimum amount of disk space to keep free on on the disk where indexed data is stored. If the limit is reached, the server stops indexing data until more space is available.
Note:
- The Splunk server will not clear any of its own disk space under this method. It will simply wait for more space to become available.
- Some events may be lost if they are not written to a file during the paused period.
In Splunk Web
- Click Admin in the upper right corner of the web interface.
- Click the Server tab.
- Click on Settings heading.
- Under the Datastore section, find Pause indexing if free disk space falls below ___ MB:
- Enter your desired minimum free disk space in megabytes.
- Click Save at the bottom of the page.
You will need to restart the server for the new setting to take effect.
From the Command line interface (CLI)
You can set the minimum free megabytes via Splunk's CLI. To use the CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command. You can also add Splunk to your path and use the splunk command.
Simply type:
# splunk set minfreemb 200000
# splunk restart
Set database size
Controls for indexes are in indexes.conf. You can control disk storage usage by controlling total index size, age of data in the database, and aging policy. When one of these limits is reached, data will be removed. You can archive the data using one of Splunk's predefined archive scripts or create your own. As with all configuration changes, you should make changes to this file in $SPLUNK_HOME/etc/bundles/local/ or create a new bundle.
Find this entry in indexes.conf
maxTotalDataSizeMB = (500000) * The maximum size of an index. If an index grows bigger than this the oldest data is frozen out. and set it to it new value (in megabytes)
Example:
[main] maxTotalDataSizeMB = 2500000
You will need to restart the server for the new setting to take effect. It will take some time, up to 30 or 40 minutes, for Splunk to move events out of the index to conform to the new policy, during which you may see high CPU usage.
This documentation applies to the following versions of Splunk: 3.2 , 3.2.1 , 3.2.2 , 3.2.3 , 3.2.4 , 3.2.5 , 3.2.6 View the Article History for its revisions.