Configure data models
Splunk Enterprise Security leverages accelerated data models to populate dashboards and views and provide correlation search results. The data models are defined and provided in the Common Information Model app (Splunk_SA_CIM), which is included in the Enterprise Security installation. Enterprise Security also installs unique data models that only apply to ES content.
Data model acceleration search load
A data model is accelerated through a scheduled summarization search process initiated on the search head. The summarization search runs on the indexers, searching newly indexed data while using the data model as a filter. The resulting matches are saved to disk alongside the index bucket for quick access.
With the release of Splunk Enterprise 6.3, there can be 2 simultaneous summarization searches running per data model, per indexer. For more information, see Parallel summarization in the Capacity Planning Manual.
Data model acceleration storage and retention
Data model acceleration uses the indexers for processing and storage, placing the accelerated data alongside each index. To calculate the additional storage needed on the indexers based on the total volume of data, use the formula:
Accelerated data model storage/year = Data volume per day * 3.4
This formula assumes that you are using the recommended retention rates for the accelerated data models.
For example, if you process 100GB/day of data volume for use with Enterprise Security, you need approximately 340GB of additional space available across all of the indexers to allow for up to one year of data model acceleration and source data retention.
Configuring storage volumes
Data model acceleration storage volumes are managed in
indexes.conf using the
tstatsHomePath parameter. The data model acceleration storage path defaults to the Splunk Enterprise default index path of
$SPLUNK_HOME/var/lib/splunk unless explicitly configured otherwise. The storage used for data model acceleration is not added to index sizing calculations for maintenance tasks such as bucket rolling and free space checks.
To manage the data model acceleration storage independently of index settings, you must define a new storage path with
[volume:] stanzas. For an example of defining a volume and storing data model accelerations, see Configure size-based retention for data models summaries in the Knowledge Manager Manual.
Data model default retention
You can change the retention settings by editing the
datamodels.conf in the app where the data model is defined.
Changing data model retention settings is contingent on the use case and data sources. A shorter retention uses less disk space and requires less processing time to maintain.
|Data Model||Summary Range||Data Model||Summary Range|
|Alerts||All Time||Application State||1 month|
|Assets And Identities (ES)||None||Authentication||1 year|
|Certificates||1 year||Change Analysis||1 year|
|Databases||None||Domain Analysis (ES)||1 year|
|1 year||Incident Management (ES)||All Time|
|Interprocess Messaging||1 year||Intrusion Detection||1 year|
|Java Virtual Machines||All Time||Network Resolution (DNS)||3 months|
|Network Sessions||3 months||Network Traffic||3 months|
|Performance||1 month||Risk Analysis (ES)||All Time|
|Splunk Audit Logs||1 year||Threat Intelligence (ES)||All Time|
|Ticket Management||1 year||Updates||1 year|
|Vulnerabilities||1 year||Web||3 months|
Data model acceleration rebuild behavior
If the configuration of the data model structure changes, or if the underlying search that creates the data model changes, the system initiates a complete rebuild of the accelerated data model. Splunk Enterprise 6.1 and later versions add a new rebuild option. As implemented in Enterprise Security, a change to the data model will not initiate an automatic rebuild. Instead, the changed data model values will apply to the newest data accelerated only. Splunk will retain any legacy data model accelerated content until the defined retention period is reached, or rolled with the index buckets.
- The rebuild configuration options are managed in the
- See Advanced configurations for persistently accelerated data models in the Knowledge Manager Manual.
- Use the Data Models management page to force a full rebuild. Navigate to Settings > Data Models, select a data model, use the left arrow to expand the row, and select the Rebuild link.
- To review the acceleration status for all data models, use the Data Model Audit dashboard.
Data model acceleration enforcement
Enterprise Security 3.0 and later enforce data model acceleration through a modular input. There are 2 ways to disable data model acceleration:
- Set the modular input to turn off Enforce Acceleration. To change the setting for a specific modular input, edit the input for the data model you are changing, uncheck the "Acceleration Enforced" setting, and save.
- Turn off our enforcement and manually edit all data model accelerations. Disable the input stanza for the data model, which will permit manual changes to a data model's acceleration settings to persist indefinitely.
Common Information Model data models
For a list of the data models are included in the Splunk Common Information Model Add-on, see What data models are included in the Common Information Model Add-on Manual.
Customized data models in Enterprise Security
In addition to the data models available as part of the Common Information Model add-on, Splunk Enterprise Security provides its own custom data models.
Configure users and roles
Planning an upgrade
This documentation applies to the following versions of Splunk® Enterprise Security: 4.1.0, 4.1.1, 4.1.2, 4.1.3, 4.1.4
Feedback submitted, thanks!