SmartStore configuration settings reside in three files:
SmartStore settings in indexes.conf
The SmartStore settings in
indexes.conf enable and control SmartStore indexes.
You can enable SmartStore for all of an indexer's indexes, or you can enable SmartStore on an index-by-index basis, allowing a mix of SmartStore and non-SmartStore indexes on the same indexer.
When you configure these settings on an indexer cluster's peer nodes, you must deploy the settings through the configuration bundle method. As with all settings in
indexes.conf, SmartStore settings must be the same across all peer nodes.
The table lists the main SmartStore-related settings in
|SmartStore indexes.conf setting||Stanza level||Description|
||index or global||Enables SmartStore and sets the remote path for the index's volume. See Deploy SmartStore on a new indexer cluster.|
||volume||Sets the volume's storage type to remote. See Deploy SmartStore on a new indexer cluster.|
||volume||Sets the remote storage location where indexes reside.
||index or global||Determines bucket freezing behavior. Sets the maximum amount of space that the warm and cold buckets of a SmartStore index can occupy. When this maximum is exceeded, the oldest bucket gets frozen. See Configure data retention for SmartStore indexes.|
||index or global||Determines bucket freezing behavior. Sets the maximum amount of cumulative raw data allowed in the warm and cold buckets of a SmartStore index. When this maximum is exceeded, the oldest bucket gets frozen. See Configure data retention for SmartStore indexes.|
||index||Specifies the time period, based on the bucket's age, that the cache manager attempts to protect a recent bucket from eviction. This setting operates on a per-index level, while the version of the setting in |
||index||Specifies the time period, based on the bucket's age, that the cache manager attempts to protect the bucket's non-journal and non-tsidx files, such as the |
indexes.conf, you can also configure various settings specific to your remote storage type. For example, there are a number of settings that begin with
remote.s3, such as
remote.s3.secret_key. These settings are specific to S3. Similarly, there are a number of settings that begin with
remote.azure. These settings are specific to GCS and Azure, respectively. You configure these settings in the stanza where you configure the remote volume.
For details on these and other
indexes.conf settings, see indexes.conf.spec.
When specifying a setting that begins with
remote.s3, you must use a lowercase "s" in "s3". Uppercase "S", such as
remote.S3... is not a valid substitute and causes any attempted update to the setting's value to not take effect.
Compress tsidx files upon upload to S3
To reduce S3 usage and improve network performance, SmartStore can compress tsidx files before uploading them to the remote store. This capability uses zstd compression. When the files are subsequently downloaded to indexers, SmartStore will automatically decompress the files before placing them in the cache.
To enable tsidx file compression, set the
remote.s3.tsidx_compression setting in
indexes.conf to true. The setting default is false.
This capability is available only for AWS S3 remote stores.
This feature is not backward compatible. If you downgrade your indexers to a pre-9.0 version after enabling tsidx compression, the indexers will be unable to use the compressed tsidx files.
If enabling this setting on an indexer cluster, you must make the change on all peer nodes through the configuration bundle method, as for any
indexes.conf setting on an indexer cluster. A restart is not required.
Consult Splunk Support before changing this setting.
Non-SmartStore-specific settings in indexes.conf
You must specify these configurations for all SmartStore indexes:
repFactor = auto, for indexes on indexer cluster peer nodes (not standalone indexers)
maxDataSize = auto. This is the default value (750MB), so you usually do not need to set it explicitly.
homePathrequires a path value. This is the location on local storage where hot and cached warm buckets reside.
coldPathrequires a path value, even though the setting is ignored except in the case of migrated indexes.
thawedPathrequires a path value, even though the setting has no practical purpose because you cannot thaw data to a SmartStore index. See Thawing data and SmartStore.
Settings in indexes.conf that are incompatible with SmartStore or otherwise restricted
SmartStore indexers and indexer clusters support a subset of features available with non-SmartStore indexers and indexer clusters. Settings related to incompatible features require special consideration.
In addition, SmartStore indexers and indexer clusters handle certain needs, such as data retention, differently from non-SmartStore indexers and indexer clusters. The associated settings are also different.
For information on features that are unsupported or restricted when using SmartStore, see:
indexes.conf settings must retain their default values:
createBloomfilter. Do not change from default of true.
enableOnlineBucketRepair. Do not change from default of true.
isReadOnly. Do not change from default of false.
enableTsidxReduction. Do not change from default of false.
maxDataSize. Do not change from default of auto (recommended).
bloomHomePath. Do not change the path location.
summaryHomePath. Do not change the path location.
tstatsHomePath. Do not change the path location.
indexes.conf settings are ignored by SmartStore:
journalCompression setting can be set to either gzip (the default) or zstd.
Path settings and SmartStore
SmartStore uses the settings
path to identify the location of index storage on the remote store.
All other path settings identify locations on the local cache, not the remote store. These settings include:
See other sections of this topic for configuration and usage notes for each of these settings.
SmartStore settings in server.conf
The SmartStore-related settings in
server.conf control the behavior of indexers, including the functionality of the cache manager. The table lists the most frequently configured settings.
|SmartStore server.conf setting||Stanza level||Description|
||Sets the policy that determines which buckets the cache manager evicts next. Defaults to "lru". See Configure the SmartStore cache manager.|
||Specifies the maximum space, in megabytes, that the cache can occupy on a disk partition. See Configure the SmartStore cache manager.|
||Specifies the additional space, in megabytes, beyond |
||Specifies the time period, based on the bucket's age, that the cache manager attempts to protect a recent bucket from eviction. This setting operates on a global level, across all indexes, while the version of the setting in |
||Specifies the time period, based on the bucket's age, that the cache manager attempts to protect the bucket's non-journal and non-tsidx files, such as the |
||Includes various, mostly low-level, settings that control remote bucket operations in indexer clusters.|
||Causes the |
For details on these and other
server.conf settings, see server.conf.spec.
Most of the SmartStore
server.conf settings, including all of the common ones, are configured on the peer nodes. A few rarely-changed settings are configured instead on the manager node. All settings configured on the peer nodes must be the same across all peer nodes.
SmartStore settings in limits.conf
limits.conf file contains several low-level search-related settings; for example,
bucket_localize_max_timeout_sec. These settings are primarily related to the process of localizing buckets. Do not change the settings without a clear understanding of the underlying processes.
For details on
limits.conf settings, see limits.conf.spec.
Bootstrap SmartStore indexes
Configure the SmartStore cache manager
This documentation applies to the following versions of Splunk® Enterprise: 9.0.0, 9.0.1, 9.0.2, 9.0.3