Extreme search is an enhancement to the Splunk Enterprise search language. As implemented in the Splunk App for Enterprise Security, you can use the Extreme search commands to:
- Build dynamic thresholds based upon event data.
- Provide context awareness by replacing event counts with natural language.
For example, in the Enterprise Security Malware Center dashboard, the Key Security Indicator Total Infections displays the total number of systems with malware infections over the last 24 hours.
The rate of change is determined by comparing the current count against the count of infected systems from the day before. The threshold is entirely user-configured. There is no automatic determination of what a normal daily range is for infected systems in the current environment. Additionally, the indicator notes an increase in infections by 3, but the value has no context. Does an increase in infected systems by 3 hosts represent a notable increase?
The same indicator using Extreme search displays the relevant information, but includes a depth that was not available with the prior Total Infections indicator.
Both the infection count and the rate of new infections are calculated using a dynamically updating model. The key security indicator is using language that is both contextual and easy to understand: The total malware infection count is not higher than any other day, and the rate of change in infections is not alarming.
The use of context and concept in Extreme search
A set of core ideas are critical to the understanding of Extreme search, and are responsible for the data model that an extreme search command uses for dynamic thresholds.
1. Concept: A concept is a semantic term that applies to data. A concept represents a qualitative description, not a numerical one. An example of a concept are the terms: "extreme," "high," "medium," "low," and "minimal".
2. Context: A context defines a relationship to a field or data in numerical terms. The data to be modeled must be represented by numerical values as the result of a search. Some examples of contexts: Total network thruput over the last 24 hours, and Network latency over the last 24 hours.
When the context and concept are combined, semantic terms can be used to answer a question about the data:
- The total network thruput over the last 24 hours was: extreme, high, medium, low, or minimal.
- The network latency over the last 24 hours was: extreme, high, medium, low, or minimal.
The terms in the concept are equally valid descriptions of network activity in both examples. Note how the semantic terms in a concept can have different meaning based upon the context it is applied to. When the environment is reporting that total network thruput is minimal, it is a warning. When the environment is reporting that network latency is minimal, then the network is operating normally.
A dynamic threshold is a context that receives updates to its data model though the use of scheduled reports.
After a context and concept are chosen to represent the data and contextual terms, a data model is created. Using the Extreme search commands, the data model maps the context and event statistics by concept. The combined model is referred to as a context when used by an Extreme search command.
A context is updated through the use of a scheduled report. The scheduled report is responsible for searching the event data for the statistics to update the context. For a list of the scheduled reports that update contexts, see Containers, contexts, and scheduled reports in this topic.
Configuring Extreme search for the Enterprise Security app
The use of Extreme search commands in the Splunk App for Enterprise Security requires no additional configuration. All contexts used by the Extreme search commands are provided in the default installation of the Enterprise Security app, and the scheduled reports to maintain them are enabled by default.
- For a list of the contexts and scheduled reports implemented in the Enterprise Security app, see Containers, contexts, and scheduled reports in this topic.
- For a list of the key security indicators that use Extreme search, see Extreme Search Key Security Indicators in this topic.
- For a list of the correlation searches that use Extreme search, see Extreme Search Correlation searches in this topic. All correlation searches are disabled by default.
Extreme Search Correlation searches
All correlation searches in the Enterprise Security app are disabled by default.
Note: Guided Search Creation is not available for correlation searches that implement Extreme search commands.
|Access - Brute Force Access Behavior Detected – Rule||failures_by_src_count_1h|
|Access - Brute Force Access Behavior Detected Over 1d - Rule||failures_by_src_count_1d|
|Change - Abnormally High Number of Endpoint Changes By User - Rule||change_count_by_user_by_change_type_1d|
|Endpoint - Host Sending Excessive Email – Rule||recipients_by_src_1h|
|Network - Substantial Increase in an Event – Rule||count_by_signature_1h|
|Network - Substantial Increase in Port Activity (By Destination) - Rule||count_by_dest_port_1d|
|Network - Unusual Volume of Network Activity - Rule||count_30m|
|Web - Abnormally High Number of HTTP Method Events By Src - Rule||count_by_http_method_by_src_1d|
Extreme Search Key Security Indicators
The indicators that use Extreme search are easily identified on a dashboard by their use of semantic language instead of numerical values. The key security indicators on each dashboard are enabled by default.
|Access - Total Access Attempts||authentication: count_1d, percentile|
|Malware - Total Infection Count||malware: count_1d, percentile|
|Risk - Median Risk Score||median_object_risk_by_object_type_1d, percentile|
|Risk - Median Risk Score By System||median_object_risk_by_object_type_1d, percentile|
|Risk - Median Risk Score By User||median_object_risk_by_object_type_1d, percentile|
|Risk - Median Risk Score By Other||median_object_risk_by_object_type_1d, percentile|
|Risk - Aggregated Risk||total_risk_by_object_type_1d, percentile|
|Risk - Aggregated System Risk||total_risk_by_object_type_1d, percentile|
|Risk - Aggregated User Risk||total_risk_by_object_type_1d, percentile|
|Risk - Aggregated Other Risk||total_risk_by_object_type_1d, percentile|
Containers, contexts, and scheduled reports
The contexts are stored in objects called containers. A container is both an object in the file system and a logical configuration for use in classifying contexts. In the Enterprise Security app, the containers are files with the
.context extension. A container can contain multiple contexts.
Note: The dynamic context search scheduled reports are enabled by default.
|Container name||Context name||App location||Dynamic context search name|
|authentication||failures_by_src_count_1h||SA-AccessProtection||Access - Authentication Failures By Source - Context Gen|
|count_1d||Access - Authentication Volume Per Day - Context Gen|
|failures_by_src_count_1d||Access - Authentication Failures By Source Per Day - Context Gen|
|change_analysis||change_count_by_user_by_change_type_1d||SA-EndpointProtection||Change - Total Change Count By User By Change Type Per Day - Context Gen|
|destinations_by_src_1h||SA-EndpointProtection||Endpoint - Emails By Destination Count - Context Gen|
|recipients_by_src_1h||Endpoint - Emails By Source - Context Gen|
|ids_attacks||count_by_signature_1h||SA-NetworkProtection||Network - Event Count By Signature Per Hour - Context Gen|
|malware||count_1d||SA-NetworkProtection||Endpoint - Malware Daily Count - Context Gen|
|network_traffic||count_by_dest_port_1d||SA-NetworkProtection||Network - Port Activity By Destination Port - Context Gen|
|count_30m||Network - Traffic Volume Per 30m - Context Gen|
|src_count_30m||Network - Traffic Source Count Per 30m - Context Gen|
|web||count_by_http_method_by_src_1d||SA-NetworkProtection||Web - Web Event Count By Src By HTTP Method Per 1d - Context Gen|
|risk||median_object_risk_by_object_type_1d||SA-ThreatIntelligence||Risk - Median Object Risk Per Day - Context Gen|
|total_risk_by_object_type_1d||Risk - Total Risk By Risk Object Type Per Day - Context Gen|
For an extended walkthrough on a correlation search that implements Extreme search commands, see "An Extreme search example" in this manual.
This documentation applies to the following versions of Splunk® Enterprise Security: 3.2, 3.2.1, 3.2.2, 3.3.0, 3.3.1, 3.3.2, 3.3.3
Feedback submitted, thanks!