Splunk® Enterprise Security

Use Splunk Enterprise Security

Download manual as PDF

This documentation does not apply to the most recent version of ES. Click here for the latest version.
Download topic as PDF

Extreme Search

Extreme search is an enhancement to the Splunk Enterprise search language. As implemented in the Splunk App for Enterprise Security, you can use the Extreme search commands to:

  • Build dynamic thresholds based upon event data.
  • Provide context awareness by replacing event counts with natural language.

For example, in the Enterprise Security Malware Center dashboard, the Key Security Indicator Total Infections displays the total number of systems with malware infections over the last 24 hours.

ES32 XS example1.png

The rate of change is determined by comparing the current count against the count of infected systems from the day before. The threshold is entirely user-configured. There is no automatic determination of what a normal daily range is for infected systems in the current environment. Additionally, the indicator notes an increase in infections by 3, but the value has no context. Does an increase in infected systems by 3 hosts represent a notable increase?

The same indicator using Extreme search displays the relevant information, but includes a depth that was not available with the prior Total Infections indicator.

ES32 XS example2.png

Both the infection count and the rate of new infections are calculated using a dynamically updating model. The key security indicator is using language that is both contextual and easy to understand: The total malware infection count is not higher than any other day, and the rate of change in infections is not alarming.

The use of context and concept in Extreme search

A set of core ideas are critical to the understanding of Extreme search, and are responsible for the data model that an extreme search command uses for dynamic thresholds.

1. Concept: A concept is a semantic term that applies to data. A concept represents a qualitative description, not a numerical one. An example of a concept are the terms: "extreme," "high," "medium," "low," and "minimal".

2. Context: A context defines a relationship to a field or data in numerical terms. The data to be modeled must be represented by numerical values as the result of a search. Some examples of contexts: Total network thruput over the last 24 hours, and Network latency over the last 24 hours.

When the context and concept are combined, semantic terms can be used to answer a question about the data:

  • The total network thruput over the last 24 hours was: extreme, high, medium, low, or minimal.
  • The network latency over the last 24 hours was: extreme, high, medium, low, or minimal.

The terms in the concept are equally valid descriptions of network activity in both examples. Note how the semantic terms in a concept can have different meaning based upon the context it is applied to. When the environment is reporting that total network thruput is minimal, it is a warning. When the environment is reporting that network latency is minimal, then the network is operating normally.

Dynamic thresholds

A dynamic threshold is a context that receives updates to its data model though the use of scheduled reports.

After a context and concept are chosen to represent the data and contextual terms, a data model is created. Using the Extreme search commands, the data model maps the context and event statistics by concept. The combined model is referred to as a context when used by an Extreme search command.

A context is updated through the use of a scheduled report. The scheduled report is responsible for searching the event data for the statistics to update the context. For a list of the scheduled reports that update contexts, see Containers, contexts, and scheduled reports in this topic.

Configuring Extreme search for the Enterprise Security app

The use of Extreme search commands in the Splunk App for Enterprise Security requires no additional configuration. All contexts used by the Extreme search commands are provided in the default installation of the Enterprise Security app, and the scheduled reports to maintain them are enabled by default.

Extreme Search Correlation searches

All correlation searches in the Enterprise Security app are disabled by default.

Note: Guided Search Creation is not available for correlation searches that implement Extreme search commands.

Search Name Context
Access - Brute Force Access Behavior Detected – Rule failures_by_src_count_1h
Access - Brute Force Access Behavior Detected Over 1d - Rule failures_by_src_count_1d
Change - Abnormally High Number of Endpoint Changes By User - Rule change_count_by_user_by_change_type_1d
Endpoint - Host Sending Excessive Email – Rule recipients_by_src_1h
Network - Substantial Increase in an Event – Rule count_by_signature_1h
Network - Substantial Increase in Port Activity (By Destination) - Rule count_by_dest_port_1d
Network - Unusual Volume of Network Activity - Rule count_30m
Web - Abnormally High Number of HTTP Method Events By Src - Rule count_by_http_method_by_src_1d

Extreme Search Key Security Indicators

The indicators that use Extreme search are easily identified on a dashboard by their use of semantic language instead of numerical values. The key security indicators on each dashboard are enabled by default.

Search Name Contexts
Access - Total Access Attempts authentication: count_1d, percentile
Malware - Total Infection Count malware: count_1d, percentile
Risk - Median Risk Score median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By System median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By User median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By Other median_object_risk_by_object_type_1d, percentile
Risk - Aggregated Risk total_risk_by_object_type_1d, percentile
Risk - Aggregated System Risk total_risk_by_object_type_1d, percentile
Risk - Aggregated User Risk total_risk_by_object_type_1d, percentile
Risk - Aggregated Other Risk total_risk_by_object_type_1d, percentile

Containers, contexts, and scheduled reports

The contexts are stored in objects called containers. A container is both an object in the file system and a logical configuration for use in classifying contexts. In the Enterprise Security app, the containers are files with the .context extension. A container can contain multiple contexts.

Note: The dynamic context search scheduled reports are enabled by default.

Container name Context name App location Dynamic context search name
authentication failures_by_src_count_1h SA-AccessProtection Access - Authentication Failures By Source - Context Gen
count_1d Access - Authentication Volume Per Day - Context Gen
failures_by_src_count_1d Access - Authentication Failures By Source Per Day - Context Gen
change_analysis change_count_by_user_by_change_type_1d SA-EndpointProtection Change - Total Change Count By User By Change Type Per Day - Context Gen
email destinations_by_src_1h SA-EndpointProtection Endpoint - Emails By Destination Count - Context Gen
recipients_by_src_1h Endpoint - Emails By Source - Context Gen
ids_attacks count_by_signature_1h SA-NetworkProtection Network - Event Count By Signature Per Hour - Context Gen
malware count_1d SA-NetworkProtection Endpoint - Malware Daily Count - Context Gen
network_traffic count_by_dest_port_1d SA-NetworkProtection Network - Port Activity By Destination Port - Context Gen
count_30m Network - Traffic Volume Per 30m - Context Gen
src_count_30m Network - Traffic Source Count Per 30m - Context Gen
web count_by_http_method_by_src_1d SA-NetworkProtection Web - Web Event Count By Src By HTTP Method Per 1d - Context Gen
risk median_object_risk_by_object_type_1d SA-ThreatIntelligence Risk - Median Object Risk Per Day - Context Gen
total_risk_by_object_type_1d Risk - Total Risk By Risk Object Type Per Day - Context Gen
default percentile SA-Utils None.
default height Splunk_SA_ExtremeSearch None.
trendchange None.
compatibility None.

Examples

For an extended walkthrough on a correlation search that implements Extreme search commands, see "An Extreme search example" in this manual.

PREVIOUS
Dashboard overview
  NEXT
Key indicators

This documentation applies to the following versions of Splunk® Enterprise Security: 3.2, 3.2.1, 3.2.2, 3.3.0, 3.3.1, 3.3.2, 3.3.3


Comments

The Network - Event Count By Signature Per Hour - Context Gen schedule is designed as a "rolling window" of the last 60 days, so the context is 'created' each time covering the last 60 days of events.

Ekostowski splunk, Splunker
May 19, 2016

Is there a reason the context gen search for Network - Event Count By Signature Per Hour - Context Gen uses a xsCreateDDContext rather than an xsUpdateDDContext ? The default is to run the search everyday at 3am and look back 60 days. Why is ES wiping out the context everyday?

Nswondem
May 18, 2016

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters