SmartStore system requirements

The requirements for SmartStore-enabled indexers are basically the same as for non-SmartStore-enabled indexers. The main differences are:

• Local storage requirements
• The need to connect to remote storage

Indexer requirements

Indexer hardware requirements

Hardware requirements, with the exception of local storage (see below), are the same as for any Splunk Enterprise indexer. See:

• System requirements and other deployment considerations for indexer clusters in this manual
• Reference hardware in Capacity Planning Manual

SmartStore operations generally have no significant impact on CPU performance.

Indexer hosting options

The type of object storage used for SmartStore determines where the indexers must be hosted:

• Indexers running SmartStore with an AWS S3 remote store must be hosted on AWS.

• Indexers running SmartStore with a GCP GCS remote store must be hosted on GCP.

• Indexers running SmartStore with an Azure Blob remote store must be hosted on Azure.

• Indexers running SmartStore with S3 API-compliant on-premises object storage must be hosted in an on-premises data center.

Local storage requirements

Based on your indexer deployment type, provision local storage as follows::

• If you are running Splunk Enterprise indexers on local Linux machines, the preferred local storage type is SSD.
• If you are running Splunk Enterprise indexers on AWS, use AWS with NVMe SSD Instance Storage (for example, AWS i3ens or i3s).
• If you are running Splunk Enterprise indexers on GCP, use N1-high memory machine type (n1-highmem-64 or n1-highmem-32 preferred) with zonal SSD persistent disks.
• If you are running Splunk Enterprise indexers on Azure, use Azure with high memory instance + SSD (E series, like Edv4 and Edsv4-series).

The amount of local storage available on each indexer for cached data must be in proportion to the expected working set. For best results, provision enough local storage to accommodate the equivalent of 30 days' worth of indexed data. For example, if the indexer is adding approximately 100GB/day of indexed data, the recommended size reserved for cached data is 3000GB. At a minimum, provision enough storage to keep at least 7-10 days of data in cache, as searches typically occur on data indexed within the last 7 - 10 days.

For use with Splunk Enterprise Security, provision enough local storage to accommodate 90 days' worth of indexed data, rather than the otherwise recommended 30 days.

Note: The size of data indexed is typically around 50% of the size of data ingested, due to compression of data during indexing. However, there are a number of other factors that also enter into determining the size ratio of indexed data to ingested data. For a general discussion of Splunk Enterprise data volume and how to estimate your storage needs, refer to "Estimating your storage requirements" in the Installation Manual.

For indexer clusters, other factors that enter into determining storage requirements include:

• The distribution of ingested data across the indexer cluster. Strive for balanced data ingestion across the indexers, so that all the indexers are indexing, and storing, approximately the same amount of data.
• The number of hot buckets on the indexers. Hot buckets follow replication and search factor policies, so they have, on a per bucket basis, larger storage requirements than warm buckets of the same size.

For purposes of configuring size, the cache is identical with the partition that the cache resides on. Typically, that partition also includes many other files, such as splunk binaries, the operating system, search artifacts, non-SmartStore indexes (if any), and so on. The index data itself resides in $SPLUNK_DB. Therfore, you must consider such files when provisioning storage and configuring cache size. For more information on cache sizing, see "Initiate eviction based on occupancy of the cache's disk partition".

Operating system requirements for on-premise deployments

If you are running Splunk Enterprise in your data center, each on-premise machine must run the same Linux 64-bit operating system.

No other operating systems are supported for on-premise deployments of SmartStore.

Splunk Enterprise version requirements

Standalone indexers must be running Splunk Enterprise version 7.2 or later.

In the case of indexer clusters, all nodes (manager, peer, and search head) in the cluster must be running Splunk Enterprise version 7.2 or later. Additional version compatibility requirements apply to nodes in any indexer cluster, as described in Splunk Enterprise version compatibility.

Other indexer cluster requirements

All indexer cluster requirements apply to SmartStore indexes. For example:

• All index-related settings, including SmartStore-settings, must be configured identically across the peer nodes.
• You must enable SmartStore for the same set of indexes on all peer nodes.
• When you add a new index stanza, you must set repFactor to "auto".

For indexer cluster requirements in general, see System requirements and other deployment considerations for indexer clusters.

Network requirements

The indexers are clients of the remote store and use the standard https port to communicate with it.

For optimal performance, use 10Gbps network connectivity from each indexer to the remote store.

Remote store requirements

SmartStore can run on either an AWS S3 remote store (including S3 API-compliant on-premise object stores), a GCP GCS remote store, or an Azure Blob remote store. Depending on which remote store type you plan to deploy on, see Configure the S3 remote store for SmartStore, Configure the GCS remote store for SmartStore, or Configure the Azure Blob remote store for SmartStore

Multiple indexer clusters or standalone indexers cannot access the same remote volume. In other words, the value of any path setting in indexes.conf must be unique to a single running indexer or cluster. Do not share path settings among multiple indexers or clusters.