Splunk® Enterprise

Admin Manual

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

How much space you will need

This topic describes how to estimate the size of your Splunk index and associated data so that you can plan your storage capacity requirements.

When Splunk indexes your data, the resulting data falls into two categories: the compressed, persisted raw data and the indexes that point to this data. With a little experimentation, you can estimate how much disk space you will need.

Typically, the compressed, persisted data amounts to approximately 10% of the raw data that comes into Splunk. The associated indexes range in size anywhere from 10% to 110% of the data they access. This value is affected strongly by the number of unique terms in the data. Depending on the data's characteristics, you might want to tune your segmentation settings.

The best way to get an idea of your space needs is to experiment by installing a copy of Splunk and indexing a representative sample of your data, and then checking the sizes of the resulting directories in defaultdb.

To do this, first index your sample. Then:

1. Go to $SPLUNK_HOME/var/lib/splunk/defaultdb/db.

2. Run du -shc hot_v*/rawdata to determine the size of the compressed, persisted raw data. Typically, this amounts to about 10% of the size of the original sample data set.

3. Run du -ch hot_v* and look at the last total line to see the size of the index.

4. Add the two values together.

This is the total size of the index and associated data for the sample you indexed. You can now use this to extrapolate the size requirements for your Splunk index and rawdata directories over time.

What you can back up
Back up indexed data

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters