Splunk® Enterprise Security

Installation and Upgrade Manual

Download manual as PDF

This documentation does not apply to the most recent version of ES. Click here for the latest version.
Download topic as PDF

Data models in the Enterprise Security app

The Splunk App for Enterprise Security leverages accelerated data models to populate dashboards and views. The data models are defined and provided in the Common Information Model app (Splunk_SA_CIM), which is included as part of the Enterprise Security app installation. Additional data models are unique to the Enterprise Security app. For a list of ES specific data models, see "Customized data models in Enterprise Security" in this topic.

Data model acceleration rebuild behavior

An accelerated data model will initiate full rebuild if data model structure changes, or if the underlying search that creates the data model changes. In Splunk Enterprise 6.1 and later, an additional rebuild option was added for data models. As implemented in the Enterprise Security app, any change to a data model will not cause an automatic rebuild of all accelerated content. The changed data model values will apply to the newest data accelerated only. The legacy data model accelerated content will be retained until the defined retention period is reached, or rolled out with the buckets.

  • Use the Data Models management page to force a full rebuild. Navigate to Settings > Data Models, select a data model, use the left arrow to expand the row, and select the Rebuild link.
  • Use the Data Model Audit dashboard to review the acceleration status for all data models.

Data model acceleration enforcement

Data model acceleration is enforced in Enterprise Security 3.0 and later through a modular input. There are 2 ways to disable data model acceleration:

  1. Set the modular input to turn off Enforce Acceleration. To change the setting for a specific modular input, edit the input for the data model you are changing, uncheck the "Acceleration Enforced" setting and save.
  2. Turn off our enforcement and manually edit all data model accelerations. Disable the input stanza for the data model, which will permit manual changes to a data model's acceleration settings to persist indefinitely.

Data model acceleration storage and retention

Data model acceleration storage volumes are managed in indexes.conf using the tstatsHomePath parameter, with the data model acceleration storage path defaulting to the Splunk Enterprise default index path $SPLUNK_HOME/var/lib/splunk unless explicitly configured. The storage used for data model acceleration is not added to index sizing calculations for maintenance tasks such as bucket rolling and free space checks.

To manage the data model acceleration storage independently of index settings, a new storage path must be defined with [volume:] stanzas. For an example of defining a volume and storing data model accelerations, see Configure size-based retention for data models summaries in the Knowledge Manager manual. Setting the retention of accelerated data models is managed in the datamodels.conf files.

Data model default retention

Data Model Summary Range Data Model Summary Range
Alerts All Time Application State 1 month
Assets And Identities (ES) None Authentication 1 year
Certificates 1 year Change Analysis 1 year
Databases None Domain Analysis (ES) 1 year
Email 1 year Incident Management (ES) All Time
Interprocess Messaging 1 year Intrusion Detection 1 year
Inventory None Malware 1 year
Java Virtual Machines All Time Network Resolution (DNS) 3 months
Network Sessions 3 months Network Traffic 3 months
Performance 1 month Risk Analysis (ES) All Time
Splunk Audit Logs 1 year Threat Intelligence (ES) All Time
Ticket Management 1 year Updates 1 year
Vulnerabilities 1 year Web 3 months

Common Information Model data models

For a list of the data models are included in the Splunk Common Information Model Add-on, see "What data models are included" in the Common Information Model Add-on Manual.

Customized data models in Enterprise Security

In addition to the data models available as part of the Common Information Model add-on, the Splunk App for Enterprise Security provides its own custom data models.


Assets And Identities

The fields in the Assets And Identities data model, and the Asset and Identity event categories, describe both asset inventory and individual account holders that should be made available across multiple Splunk application contexts.

Note: Any field in the All_Assets event category can be optionally pre-pended with dest_, dvc_, host_, orig_host_, or src_ for enrichment purposes. These fields are not required, but are often used in Apps alongside dest, dvc, host, orig_host, or src if they are available.

Tags are not applicable to the Asset And Identities data model and event category.

Fields for the Asset And Identities data model and event category

Object name(s) Field name Data type Description Expected values
All_Assets asset_id string an identifier for the asset, such as an asset tag or serial number.
All_Assets city string The city where the asset is located, such as San Francisco.
All_Assets bunit string The business unit of the asset, such as Marketing.
All_Assets category MV string The category of the asset, such as email_server or SOX-compliant.
All_Assets country string The country where the asset is located, such as USA.
All_Assets dns MV string A fully qualified domain name (FQDN) associated with the asset, such as server42.splunk.com.
All_Assets ip MV string An IP address (either v4 or v6) associated with the asset, such as 192.168.4.2. Note: Please remove zero-padding on this field.
All_Assets is_expected boolean A flag indicating whether the asset is expected to continually send data to Splunk. Note: Some apps may alert if is_expected is set to Y for an asset that is not sending data. true, false
All_Assets lat string The latitude of an asset's location.
All_Assets location string The physical location of an asset.
All_Assets long string The longitude of an asset's location.
All_Assets mac MV string A MAC address associated with the asset, such as 06:10:9f:eb:8f:14. Note: Always force lower case on this field. Note: Always use colons instead of dashes, spaces, or no separator.
All_Assets nt_host string The cross-platform short name or NetBIOS name of the asset, such as server42. Note: Always force lower case on this field.
All_Assets owner MV string The owner of the asset, such as jdoe.
All_Assets priority string The priority of the asset. critical, high, medium, low, informational, unknown
All_Assets requires_av boolean Flag that indicates whether the asset is expected to use a local antivirus or endpoint protection tool. Note that some apps may alert if requires_av is set to true for an asset that is not running an antivirus service and/or does not have event types properly configured for that service. true, false
All_Assets should_timesync boolean Flag that indicates whether the asset is expected to maintain time synchronization. Note that some apps may alert if should_timesync is set to true for an asset that is not running a time synchronization service and/or does not have event types properly configured for that service. true, false
All_Assets should_update boolean Flag that indicates whether the asset is expected to regularly apply patches. Note that some apps may alert if should_update is set to true for an asset that is not running a patching service and/or does not have event types properly configured for that service. true, false
All_Identities bunit string The business unit of the identity, such as Sales.
All_Identities category MV string The category of the identity, such as sales or customer_facing.
All_Identities city string The city where the identity is based, such as San Francisco.
All_Identities country string The country where the identity is based, such as USA.
All_Identities email MV string The email address (or addresses) associated with the identity is based. Note that this is a multivalue field.
All_Identities end_date timestamp The end date of the identity, leave blank if not applicable. Note that presence of an end_date in the past may cause some Apps to create alerts from events involving this identity.
All_Identities first string A first name for the identity, such as Jane.
All_Identities identity MV string Account names and numbers associated with the identity. Note that this is a multivalue field.
All_Identities last string A last name for the identity, such as Doe.
All_Identities lat string The latitude of the identity's base location.
All_Identities location string The base location for the identity, such as an office name.
All_Identities long string The longitude of the identity's base location.
All_Identities managed_by MV string The manager(s) of the identity such as jdoe. Note that this is a multivalue field and should use account names or numbers from the identity field.
All_Identities nick string A nickname for the identity, such as Moerex.
All_Identities phone MV string A phone number (or set of phone numbers) for the identity. Note that this is a multivalue field.
All_Identities phone2 MV string A phone number (or set of phone numbers) for the identity. Note that this is a multivalue field.
All_Identities prefix string A prefix for the identity, such as Mr..
All_Identities priority string The priority of the identity. critical, high, medium, low, informational, unknown
All_Identities start_date timestamp The start date of the identity.
All_Identities suffix string A suffix for the identity, such as Jr.
All_Identities watchlist boolean Flag if the identity is on a watchlist. Note that some apps may create alerts for events that involve this identity if this flag is set. true, false


Domain Analysis

The Domain Analysis data model is available as part of the SA-NetworkProtection add-on, included with the Splunk App for Enterprise Security. Domain Analysis data model search searches for index=whois sourcetype=Whois:*.

The fields and tags in the Domain Analysis data model describe the domain information in your deployment.

Tags used with the Domain Analysis data model

Object name(s) Tag name Required?
All_Domains index=whois sourcetype=Whois:* YES

Fields for the Domain Analysis data model and event category

Object name(s) Field name Data type Description Expected values
All_Domains domain string name of the domain
All_Domains nameservers string name of the server associates with this domain
All_Domains registrant string
All_Domains registrar string
All_Domains resolved_domain string resolved domain name

Incident Management

The Incident Management data model is available as part of the SA-ThreatIntelligence add-on, included with the Splunk App for Enterprise Security. This data model reads from index=notable.

The fields in the Incident Management event category describe events gathered by network monitoring devices and apps.

Tags used with the Incident Management event category

Object name(s) Tag name or constraint Required?
Notable_Events (Metatdata only) index=notable YES

Fields for the Incident Management data model

Object name(s) Field name Data type Description Possible values
Notable_Events_Meta tag string
Notable_Events_Meta rule_id string
Notable_Events_Meta decoration string
Correlation_Searches control string
Correlation_Searches default_owner string
Correlation_Searches default_status string
Correlation_Searches description string
Correlation_Searches governance string
Correlation_Searches rule_name string
Correlation_Searches saved_search string
Correlation_Searches security_domain string
Correlation_Searches severity string
Incident Review comment string
Incident Review owner string
Incident Review reviewer string
Incident Review rule_id string
Incident Review security_domain string
Incident Review status_group string
Incident Review status_label string
Incident Review tag string
Incident Review urgency string
Notable_Events dest string
Notable_Events owner string
Notable_Events owner_realname string
Notable_Events rule_name string
Notable_Events security_domain string
Notable_Events source string
Notable_Events src string
Notable_Events status_label string
Notable_Events status_group string
Notable_Events tag string
Notable_Events urgency string
Notable_Owners owner string
Notable_Owners owner_realname string
Review_Statuses default boolean
Review_Statuses end boolean
Review_Statuses hidden boolean
Review_Statuses status string
Review_Statuses status_description string
Review_Statuses status_label string
Security_Domains is_enabled boolean
Security_Domains is_expected boolean
Security_Domains is_ignored boolean
Security_Domains security_domain_label string
Suppression_Audit action string
Suppression_Audit signature string
Suppression_Audit status string
Suppression_Audit suppression string
Suppression_Audit user string
Suppression_Audit_Expired suppression string
Suppression_Eventtypes description string
Suppression_Eventtypes disabled boolean
Suppression_Eventtypes end_time timestamp
Suppression_Eventtypes search string
Suppression_Eventtypes suppression string
Suppression_Eventtypes start_time timestamp
Suppressed_Notable_Events dest string
Suppressed_Notable_Events rule_name string
Suppressed_Notable_Events security_domain string
Suppressed_Notable_Events signature string
Suppressed_Notable_Events source string
Suppressed_Notable_Events suppression string
Suppressed_Notable_Events tag string
Suppressed_Notable_Events urgency string
Urgencies priority string
Urgencies severity string
Urgencies urgency string

Risk Analysis

Object name(s) Field name Data type Description Expected values
All_Risk description string A short description of the correlation search that generated the risk modifier. calculated
All_Risk risk_object string The value of the object this modifier applies to. src,dest,etc.
All_Risk risk_object_type string The object type this modifier applies to. system,user,other,etc.
All_Risk risk_score integer The amount of "points" to increase or decrease the risk_object's score by.

Threat Intelligence

The fields and tags in the Threat Intelligence data model describe key fields found in events that match against threat list content. The Threat Lists data model is custom to the Enterprise Security app, and is defined in the SA-ThreatIntelligence add-on.

Constraints for Threat Intelligence event objects

The relevant constraints to identify events that belong in this data model.

Object name Constraint
Threat_Intelligence index=threat_activity

Fields for Threat Intelligence event objects

The following table lists the extracted and calculated fields for the event objects in the model. Note that it does not include any inherited fields. For more information, see "How to use these reference tables" in the Common Information Model Add-on Manual.

Object name Field name Data type Description
Threat_Activity dest_bunit string
Threat_Activity dest_category string
Threat_Activity dest_priority string
Threat_Activity src_bunit string
Threat_Activity src_category string
Threat_Activity src_priority string
Threat_Activity threat_match_field string
Threat_Activity threat_match_value string
Threat_Activity threat_collection string
Threat_Activity threat_collection_key string
Threat_Activity threat_key string
PREVIOUS
Dashboard requirements matrix
  NEXT
List of Indexes

This documentation applies to the following versions of Splunk® Enterprise Security: 3.3.0, 3.3.1, 3.3.2, 3.3.3


Comments

Hello Jstoner.
Yes, there's a plan to prove that material in a future release. Thanks!

Ekostowski splunk, Splunker
September 30, 2015

Are there plans to populate the threat intel model descriptions and the incident management data model descriptions?

Jstoner splunk, Splunker
September 29, 2015

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters