Use WORM (Write Once Read Many) volumes for Splunk's datastore
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
Use WORM (Write Once Read Many) volumes for Splunk's datastore
Splunk can use WORM (Write Once Read Many) volumes for its index datastores. There are a few configurations that must be made in indexes.conf in order to use WORM volumes. When using a WORM volume for index datastore, Splunk will need to be configured to push data to its warm, and cold databases differently.
In a typical Splunk datastore configuration (with multiple-write disks), Splunk manages its index datastore by reading and writing into the hot database. It then pushes data to the warm database, where it is written and read multiple times. Finally, it pushes data to the cold database, where is it written once and stored until it is pushed to frozen (only requiring reads to do so).
In a write-once datastore scenario, data from the index is never pushed to the warm database. Data is pushed directly to the cold database because it is written once and never required to be written again.
Set up a WORM datastore
- First, determine the specifications for your datastore.
- Take a look at your data retention policy. Determine how long, how much, and any other data retention specification you need to take into account when configuring a datastore.
- Next, figure out how much data you will be passing into Splunk. 500MB/day? 50GB/day?
- Use that information to determine the size and number of buckets in your datastore (example: 20GB/day retained for 30days = 60 buckets..etc.). This is how many buckets you will need in your cold database.
- Next, edit your
indexes.conffile (You should make changes to this file in$SPLUNK_HOME/etc/bundles/localor create a new bundle.).- Set the
maxWarmDBCount = 0to keep data from going into the warm database (failure to do so will cause Splunk to crash in a WORM configuration). - Set
maxColdDBCount = <number of buckets you need>to a number greater than the anticipated number of buckets.
- Set the
- Mount your WORM to the location of the cold database. Set the path to:
$SPLUNK_HOME/var/lib/splunk/defaultdb/cold.
This documentation applies to the following versions of Splunk: 3.0 , 3.0.1 , 3.0.2 , 3.1 , 3.1.1 , 3.1.2 , 3.1.3 , 3.1.4 View the Article History for its revisions.