Use WORM (Write Once Read Many) volumes for Splunk's datastore

Use WORM (Write Once Read Many) volumes for Splunk's datastore

Configure Splunk to use WORM (Write Once Read Many) volumes for its indexes by editing indexes.conf. Edit this file in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work.

Note: To use WORM volume for indexes, you must configure Splunk to push data to its warm, and cold databases differently.

In a typical Splunk index configuration (with multiple-write disks), Splunk manages its indexes by reading and writing into the hot database. It then pushes data to the warm database, where it is written and read multiple times. Finally, it pushes data to the cold database, where is it written once and stored until it is pushed to frozen.

In a write-once setup, data from the index never goes to the warm database. Data goes from hot directly to the cold database because it is written once and never required to be written again.

Configuration

• Determine data retention specifications.
• Next, figure out how much data you will be passing into Splunk. 500MB/day? 50GB/day?
• Use that information to determine the size and number of buckets in your indexes (example: 20GB/day retained for 30days = 60 buckets). This is how many buckets you will need in your cold database.
• Next, edit the following attributes/values in indexes.conf:

[<index name>]
maxWarmDBCount = 0
maxColdDBCound = <number of buckets>

• Set maxWarmDBCount = 0 to keep data from going into the warm database (failure to do so will cause Splunk to crash in a WORM configuration).
• Set maxColdDBCount to a number greater than the anticipated number of buckets.
• Mount your WORM to the location of the cold database. Set the path to: $SPLUNK_HOME/var/lib/splunk/defaultdb/cold.

• Was this topic useful?
• Post a comment