Extreme Search

Extreme search is an enhancement to the Splunk Enterprise search language. As implemented in the Splunk App for Enterprise Security, you can use the Extreme search commands to:

• Build dynamic thresholds based upon event data.
• Provide context awareness by replacing event counts with natural language.

For example, in the Enterprise Security Malware Center dashboard, the Key Security Indicator Total Infections displays the total number of systems with malware infections over the last 24 hours.

The rate of change is determined by comparing the current count against the count of infected systems from the day before. The threshold is entirely user-configured. There is no automatic determination of what a normal daily range is for infected systems in the current environment. Additionally, the indicator notes an increase in infections by 3, but the value has no context. Does an increase in infected systems by 3 hosts represent a notable increase?

The same indicator using Extreme search displays the relevant information, but includes a depth that was not available with the prior Total Infections indicator.

Both the infection count and the rate of new infections are calculated using a dynamically updating model. The key security indicator is using language that is both contextual and easy to understand: The total malware infection count is not higher than any other day, and the rate of change in infections is not alarming.

The use of context and concept in Extreme search

A set of core ideas are critical to the understanding of Extreme search, and are responsible for the data model that an extreme search command uses for dynamic thresholds.

1. Concept: A concept is a semantic term that applies to data. A concept represents a qualitative description, not a numerical one. An example of a concept are the terms: "extreme," "high," "medium," "low," and "minimal".

2. Context: A context defines a relationship to a field or data in numerical terms. The data to be modeled must be represented by numerical values as the result of a search. Some examples of contexts: Total network thruput over the last 24 hours, and Network latency over the last 24 hours.

When the context and concept are combined, semantic terms can be used to answer a question about the data:

• The total network thruput over the last 24 hours was: extreme, high, medium, low, or minimal.
• The network latency over the last 24 hours was: extreme, high, medium, low, or minimal.

The terms in the concept are equally valid descriptions of network activity in both examples. Note how the semantic terms in a concept can have different meaning based upon the context it is applied to. When the environment is reporting that total network thruput is minimal, it is a warning. When the environment is reporting that network latency is minimal, then the network is operating normally.

Dynamic thresholds

A dynamic threshold is a context that receives updates to its data model though the use of scheduled reports.

After a context and concept are chosen to represent the data and contextual terms, a data model is created. Using the Extreme search commands, the data model maps the context and event statistics by concept. The combined model is referred to as a context when used by an Extreme search command.

A context is updated through the use of a scheduled report. The scheduled report is responsible for searching the event data for the statistics to update the context. For a list of the scheduled reports that update contexts, see Containers, contexts, and scheduled reports in this topic.

Configuring Extreme search for the Enterprise Security app

The use of Extreme search commands in the Splunk App for Enterprise Security requires no additional configuration. All contexts used by the Extreme search commands are provided in the default installation of the Enterprise Security app, and the scheduled reports to maintain them are enabled by default.

• For a list of the contexts and scheduled reports implemented in the Enterprise Security app, see Containers, contexts, and scheduled reports in this topic.

• For a list of the key security indicators that use Extreme search, see Extreme Search Key Security Indicators in this topic.

• For a list of the correlation searches that use Extreme search, see Extreme Search Correlation searches in this topic. All correlation searches are disabled by default.

Extreme Search Correlation searches

All correlation searches in the Enterprise Security app are disabled by default.

Note: Guided Search Creation is not available for correlation searches that implement Extreme search commands.

Search Name	Context
Access - Brute Force Access Behavior Detected – Rule	failures_by_src_count_1h
Access - Brute Force Access Behavior Detected Over 1d - Rule	failures_by_src_count_1d
Change - Abnormally High Number of Endpoint Changes By User - Rule	change_count_by_user_by_change_type_1d
Endpoint - Host Sending Excessive Email – Rule	recipients_by_src_1h
Network - Substantial Increase in an Event – Rule	count_by_signature_1h
Network - Substantial Increase in Port Activity (By Destination) - Rule	count_by_dest_port_1d
Network - Unusual Volume of Network Activity - Rule	count_30m
Web - Abnormally High Number of HTTP Method Events By Src - Rule	count_by_http_method_by_src_1d

Extreme Search Key Security Indicators

The indicators that use Extreme search are easily identified on a dashboard by their use of semantic language instead of numerical values. The key security indicators on each dashboard are enabled by default.

Search Name	Contexts
Access - Total Access Attempts	authentication: count_1d, percentile
Malware - Total Infection Count	malware: count_1d, percentile
Risk - Median Risk Score	median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By System	median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By User	median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By Other	median_object_risk_by_object_type_1d, percentile
Risk - Aggregated Risk	total_risk_by_object_type_1d, percentile
Risk - Aggregated System Risk	total_risk_by_object_type_1d, percentile
Risk - Aggregated User Risk	total_risk_by_object_type_1d, percentile
Risk - Aggregated Other Risk	total_risk_by_object_type_1d, percentile

Containers, contexts, and scheduled reports

The contexts are stored in objects called containers. A container is both an object in the file system and a logical configuration for use in classifying contexts. In the Enterprise Security app, the containers are files with the .context extension. A container can contain multiple contexts.

Note: The dynamic context search scheduled reports are enabled by default.

Container name	Context name	App location	Dynamic context search name
authentication	failures_by_src_count_1h	SA-AccessProtection	Access - Authentication Failures By Source - Context Gen
	count_1d		Access - Authentication Volume Per Day - Context Gen
	failures_by_src_count_1d		Access - Authentication Failures By Source Per Day - Context Gen
change_analysis	change_count_by_user_by_change_type_1d	SA-EndpointProtection	Change - Total Change Count By User By Change Type Per Day - Context Gen
email	destinations_by_src_1h	SA-EndpointProtection	Endpoint - Emails By Destination Count - Context Gen
	recipients_by_src_1h		Endpoint - Emails By Source - Context Gen
ids_attacks	count_by_signature_1h	SA-NetworkProtection	Network - Event Count By Signature Per Hour - Context Gen
malware	count_1d	SA-NetworkProtection	Endpoint - Malware Daily Count - Context Gen
network_traffic	count_by_dest_port_1d	SA-NetworkProtection	Network - Port Activity By Destination Port - Context Gen
	count_30m		Network - Traffic Volume Per 30m - Context Gen
	src_count_30m		Network - Traffic Source Count Per 30m - Context Gen
web	count_by_http_method_by_src_1d	SA-NetworkProtection	Web - Web Event Count By Src By HTTP Method Per 1d - Context Gen
risk	median_object_risk_by_object_type_1d	SA-ThreatIntelligence	Risk - Median Object Risk Per Day - Context Gen
	total_risk_by_object_type_1d		Risk - Total Risk By Risk Object Type Per Day - Context Gen
default	percentile	SA-Utils	None.
default	height	Splunk_SA_ExtremeSearch	None.
	trendchange		None.
	compatibility		None.

Examples

For an extended walkthrough on a correlation search that implements Extreme search commands, see "An Extreme search example" in this manual.