Configure data models

Splunk Enterprise Security leverages accelerated data models to populate dashboards and views and provide correlation search results. The data models are defined and provided in the Common Information Model app (Splunk_SA_CIM), which is included in the Enterprise Security installation. Enterprise Security also installs unique data models that only apply to ES content.

Data model acceleration search load

A data model is accelerated through a scheduled summarization search process initiated on the search head. The summarization search runs on the indexers, searching newly indexed data while using the data model as a filter. The resulting matches are saved to disk alongside the index bucket for quick access.

With the release of Splunk Enterprise 6.3, there can be 2 simultaneous summarization searches running per data model, per indexer. For more information, see Parallel summarization in the Capacity Planning Manual.

Data model acceleration storage and retention

Data model acceleration uses the indexers for processing and storage, placing the accelerated data alongside each index. To calculate the additional storage needed on the indexers based on the total volume of data, use the formula:

Accelerated data model storage/year = Data volume per day * 3.4

This formula assumes that you are using the recommended retention rates for the accelerated data models.

For example, if you process 100GB/day of data volume for use with Enterprise Security, you need approximately 340GB of additional space available across all of the indexers to allow for up to one year of data model acceleration and source data retention.

Configuring storage volumes

Data model acceleration storage volumes are managed in indexes.conf using the tstatsHomePath parameter. The data model acceleration storage path defaults to the Splunk Enterprise default index path of $SPLUNK_HOME/var/lib/splunk unless explicitly configured otherwise. The storage used for data model acceleration is not added to index sizing calculations for maintenance tasks such as bucket rolling and free space checks.

To manage the data model acceleration storage independently of index settings, you must define a new storage path with [volume:] stanzas. For an example of defining a volume and storing data model accelerations, see Configure size-based retention for data models summaries in the Knowledge Manager Manual.

Data model default retention

You can change the retention settings by editing the datamodels.conf in the app where the data model is defined.

Changing data model retention settings is contingent on the use case and data sources. A shorter retention uses less disk space and requires less processing time to maintain.

Data Model	Summary Range	Data Model	Summary Range
Alerts	All Time	Application State	1 month
Assets And Identities (ES)	None	Authentication	1 year
Certificates	1 year	Change Analysis	1 year
Databases	None	Domain Analysis (ES)	1 year
Email	1 year	Incident Management (ES)	All Time
Interprocess Messaging	1 year	Intrusion Detection	1 year
Inventory	None	Malware	1 year
Java Virtual Machines	All Time	Network Resolution (DNS)	3 months
Network Sessions	3 months	Network Traffic	3 months
Performance	1 month	Risk Analysis (ES)	All Time
Splunk Audit Logs	1 year	Threat Intelligence (ES)	All Time
Ticket Management	1 year	Updates	1 year
Vulnerabilities	1 year	Web	3 months

Data model acceleration rebuild behavior

If the configuration of the data model structure changes, or if the underlying search that creates the data model changes, the system initiates a complete rebuild of the accelerated data model. Splunk Enterprise 6.1 and later versions add a new rebuild option. As implemented in Enterprise Security, a change to the data model will not initiate an automatic rebuild. Instead, the changed data model values will apply to the newest data accelerated only. Splunk will retain any legacy data model accelerated content until the defined retention period is reached, or rolled with the index buckets.

• The rebuild configuration options are managed in the datamodels.conf file.

• See Advanced configurations for persistently accelerated data models in the Knowledge Manager Manual.

• Use the Data Models management page to force a full rebuild. Navigate to Settings > Data Models, select a data model, use the left arrow to expand the row, and select the Rebuild link.

• To review the acceleration status for all data models, use the Data Model Audit dashboard.

Data model acceleration enforcement

Enterprise Security 3.0 and later enforce data model acceleration through a modular input. There are 2 ways to disable data model acceleration:

1. Set the modular input to turn off Enforce Acceleration. To change the setting for a specific modular input, edit the input for the data model you are changing, uncheck the "Acceleration Enforced" setting, and save.
2. Turn off our enforcement and manually edit all data model accelerations. Disable the input stanza for the data model, which will permit manual changes to a data model's acceleration settings to persist indefinitely.

Common Information Model data models

For a list of the data models are included in the Splunk Common Information Model Add-on, see What data models are included in the Common Information Model Add-on Manual.

Customized data models in Enterprise Security

In addition to the data models available as part of the Common Information Model add-on, Splunk Enterprise Security provides its own custom data models.