An Extreme search example

You can convert existing correlation searches to use Extreme search commands. In this example, Splunk Enterprise Security includes the converted search. You do not need to make any configuration changes or modifications to use searches converted to use Extreme search.

The Brute Force Access Behavior Detected search

The correlation search "Brute Force Access Behavior Detected" searches for an excessive number of failed login attempts, followed by a successful attempt. The base search finds relevant events, counts the events by type "failure" and looks for a trailing "success" event for every host authentication over the last hour. If the identified events meet a threshold, the search triggers an alert action to create a notable event or other alert types.

"Brute Force Access Behavior Detected" correlation search without Extreme search commands:

| `datamodel("Authentication","Authentication")` | stats values(Authentication.tag) as tag,count(eval('Authentication.action'=="failure")) as failure,count(eval('Authentication.action'=="success")) as success by Authentication.src | `drop_dm_object_name("Authentication")` | search failure>6 success>0 | `settags("access")`

Without Extreme search commands, the search defines a static threshold for the "success" events with the string | search failure>6. The Enterprise Security administrator has to select a threshold value, or accept the default value. If the administrator sets the threshold too low, the search creates a storm of notable events. If they set the threshold too high, the search could miss notable events, creating a potential blind spot to a security threat.

A search that implements Extreme search removes the static value and uses, in this example, the authentication data ingested by Splunk Enterprise to determine a notable level of authentication failures in your environment.

1. Examine the data

To use Extreme search, you must build a data model for the commands to rely on. To build the data model, you must understand what the data represents and what question you are trying to answer.

In this example, the "Brute Force Access Behavior Detected" correlation search, you know that the count of authentication failures will not go below zero, and may range much higher. A scale of magnitude represents the authentication values being searched.

2. Choose a context

You can choose one of three types of contexts, each requiring three data points.

• Median or mean average: requires a median or a mean value, a standard deviation, and a total count of events.
• Domain: requires a minimum, a maximum, and a total count of events.

In this example, the count of authentication events does not include a negative value and is progressive, so a domain is the best fit for the authentication data.

3. Choose a concept

A concept represents a qualitative description of the data. Splunk Enterprise Security includes predefined concepts for interpreting change, direction, and magnitude as a qualitative value. Concepts are differentiated by the terms used.

• Change uses the terms: "minimally, slightly, moderately, greatly, extremely."
• Direction uses the terms: "decreasing, unchanged, increasing"
• Magnitude uses the terms: "minimal, low, medium, high, extreme"

In this case, the magnitude concept best represents the behavior of authentication failures.

4. Create the context

As described in Extreme Search in this manual, a context has both a name and a container, with the container residing in an app. The "Brute Force Access Behavior Detected" search runs against authentication events, so the context container is called "authentication." The "authentication" container is located in the "SA-AccessProtection" app along with the authentication searches and other objects.

ES includes a pre-initialized authentication context. This context will not represent your environment unless a saved search updates it with events. Splunk Enterprise Security contains this context so that updates will carry a greater weight than the values used during the creation of the context. The domain for this authentication context is defined with a min=0, max=10, and count=0.

For the "Brute Force Access Behavior Detected" search, the context name is chosen to facilitate quick identification: failures_by_src_count_1h.

Create the initial content:

| xsCreateUDContext app="SA-AccessProtection" name=failures_by_src_count_1h container=authentication scope=app terms=`xs_default_magnitude_concepts` min=0 max=10 count=0 type=domain

Display the context, once created:

| xsdisplaycontext failures_by_src_count_1h in authentication

Before implementing Extreme search, the static threshold for authentication failures was six. Using the context failures_by_src_count_1h, a count of six is modeled at the end of the term "medium". The model will change after the updated "Brute Force Access Behavior Detected" search searches the authentication data and the saved search that updates the failures_by_src_count_1h runs.

List the terms used in a context:

| xslistconcepts failures_by_src_count_1h in authentication

5. Apply the context in the search

You can use the search command xsWhere to evaluate a data value against a context. This correlation search uses xsWhere to compare the count of authentication failures against the context failures_by_src_count_1h to determine if the count represents a value above "medium."

In this example, a concept of medium represents the range of values that change after the context is updated with data. A saved search updates the context. If the count of events identified by the saved search is greater than medium, the correlation search using extreme search will trigger an alert action and create a notable event.

"Brute Force Access Behavior Detected" with Extreme search capabilities

| `datamodel("Authentication","Authentication")` | stats values(Authentication.tag) as tag,count(eval('Authentication.action'=="failure")) as failure,count(eval('Authentication.action'=="success")) as success by Authentication.src | `drop_dm_object_name("Authentication")` | search success>0 | xswhere failure from failures_by_src_count_1h in authentication is above medium | `settags("access")`

6. Update the context

A search threshold can be dynamic because it uses a saved search to update a context. The saved searches included with ES that generate context information for Extreme search end with "Context Gen" to provide easy identification.

The domain context used by the "Brute Force Access Behavior" correlation search requires values for minimum, maximum, and count. Those values are drawn from the authentication data model. The "Access - Authentication Failures By Source - Context Gen" saved search that generates the failures_by_src_count_1h context for the "Brute Force Access Behavior" correlation search.

For the failures_by_src_count_1h context, the results of the context generating search change the maximum value to a multiple of the median to prevent outliers from skewing the underlying context and potentially introducing oversights.

"Access - Authentication Failures By Source - Context Gen" saved search

| tstats `summariesonly` count as failures from datamodel=Authentication where Authentication.action="failure" by Authentication.src,_time span=1h | stats median(failures) as median, min(failures) as min, count as count | eval max = median*2 | xsUpdateDDContext app="SA-AccessProtection" name=failures_by_src_count_1h container=authentication scope=app

This search updates the failures_by_src_count_1h context with xsUpdateDDContext. In this case, the data from the search is added to the context, creating a historical trend that informs the context.

Both the correlation search and the saved search "Access - Authentication Failures By Source - Context Gen" are scheduled to run hourly by default.

7. Use hedges to modify the results

Hedges are semantic terms that modify the range represented by a concept. Use a hedge to limit, shrink, or modify the shape of the curve that a concept term uses to model the data. The hedges "above" and "below" are useful for alerting searches as they redefine the range of values that will match.

The "Brute Force Access Behavior Detected" correlation search using extreme search applies a hedge so an alert action triggers only when the count of failures is "above medium."

Examples of a concept with various hedges applied:

Hedge example	Image
| xsDisplayConcept medium from failures_by_src_count_1h in authentication
| xsDisplayConcept very medium from failures_by_src_count_1h in authentication
| xsDisplayConcept above medium from failures_by_src_count_1h in authentication
| xsDisplayConcept below medium from failures_by_src_count_1h in authentication
| xsDisplayConcept around medium from failures_by_src_count_1h in authentication

The synonyms.csv lookup file in the Splunk_SA_ExtremeSearch app contains the Extreme search hedges.

Summary

The "Brute Force Access Behavior Detected" correlation search using Extreme search is included with Splunk Enterprise Securirty. The context generation search runs and updates the context on a recurring interval. The correlation search references the context, and the concept within the context sets the threshold. The concept is hedged to "above medium" so that the correlation search will only create a notable event when the count of failed authentications followed by a successful authentication is "high" or "extreme."

In plain language, Extreme search transformed the "Brute Force Access Behavior Detected" correlation search from "find all authentication attempts where X count of failed authentications are followed by a successful authentication" to "find all authentication attempts where a high or extreme number of failed authentications are followed by a successful authentication."

Extreme search commands

Search command	Description
xsWhere	Used to match a concept within a specified context, and determine compatibility.
xsFindBestConcept	Used when evaluating a search count and comparing the count to a context. The closest match returns the term used by the concept. The key security indicators use this command.
xsUpdateDDContext	Used to update a data-defined context. A scheduled report that calls "xsUpdateDDContext" builds a context that represents a historical view. |xsUpdateDDContext in app=<app> name=<context> container=<container> scope=app
xsListContexts	Used to list all contexts in a container | xsListContexts in <container>
xsListConcepts	Used to list all concepts in a context | xsListConcepts from <context> in <container>
xsDisplayContext	Used to display the range of values in a context, including the terms used in the concept: | xsDisplayContext <context> IN <container>
xsDisplayConcept	Used to display the range of values used for a concept: | xsDisplayConcept <concept> from <context> in <container> | xsDisplayConcept <hedge> <concept> from <context> in <container>