Manage data integrity
The Splunk Enterprise data integrity control feature provides a way to verify the integrity of data that is indexed.
When you enable data integrity control for an index, Splunk Enterprise computes hashes (using SHA 256) on every slice of data and stores those hashes so that you can go back later and verify the integrity of your data.
How it works
When you enable data integrity control, Splunk Enterprise computes hashes on every slice of newly indexed raw data and writes it to a
l1Hashes file. When the bucket rolls from hot to warm, Splunk Enterprise computes a hash on the contents of the
l1Hashes and stores the computed hash in
l2Hash. Both hash files are stored in the
rawdata directory for that bucket.
Note that data integrity control hashes newly indexed data, data coming from a forwarder should be secured and encrypted with SSL. For more information, see About securing Splunk with SSL.
Check your hashes to validate your data
To check Splunk Enterprise data, run the following CLI command to verify the integrity of an index or bucket:
./splunk check-integrity -bucketPath [ bucket path ] [ -verbose ] ./splunk check-integrity -index [ index name ] [ -verbose ]
Configure data integrity control
To configure Data Integrity Control, edit
indexes.conf to enable the
enableDataIntegrityControl attribute for each index. The default value for all indexes is
Data Integrity in clustered environments
In a clustered environment, the cluster master and all the peers must run Splunk Enterprise 6.3 or later to enable accurate index replication.
Optionally modify the size of your data slice
By default, data slices are set to 128kb, which means that a data slice is created and hashed every 128KB. You can optionally edit
indexes.conf to specify the size of each slice.
rawChunkSizeBytes = 131072
Store and secure your data hashes
For optimal security, you can optionally store your hashes outside the system where the data is hosted, such as a different server. To avoid naming conflicts, store your secured hashes in separate directories.
If you lose your hashes for a bucket, Use the following CLI command to re-generate hash files on a bucket or index. This command extracts the hashes embedded in the journal:
./splunk generate-hash-files -bucketPath [ bucket path ] [ verbose ] ./splunk generate-hash-files -index [ index name ] [ verbose ]
Use audit events to secure Splunk Enterprise
Avoid malicious CSV files in searches
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4