Splunk Cloud

Search Reference

Download manual as PDF

Download topic as PDF

anomalies

Description

Use the anomalies command to look for events or field values that are unusual or unexpected.

The anomalies command assigns an unexpectedness score to each event and places that score in a new field named unexpectedness. Whether the event is considered anomalous or not depends on a threshold value. The threshold value is compared to the unexpectedness score. The event is considered unexpected or anomalous if the unexpectedness score is greater than the threshold value.

After you use the anomalies command in a search, look at the Interesting Fields list in the Search & Reporting window. Select the unexpectedness field to see information about the values in your events.

The unexpectedness score of an event is calculated based on the similarity of that event (X) to a set of previous events (P).

The formula for unexpectedness is:

unexpectedness =  [s(P and X) - s(P)] / [s(P) + s(X)]

In this formula, s( ) is a metric of how similar or uniform the data is. This formula provides a measure of how much adding X affects the similarity of the set of events. The formula also normalizes the results for the differing event sizes.

Syntax

anomalies [threshold=num] [labelonly=bool] [normalize=bool] [maxvalues=int] [field=field] [blacklist=filename] [blacklistthreshold=num] [by-clause]

Optional arguments

threshold
Datatype: threshold=<num>
Description: A number to represent the upper limit of expected or normal events. If unexpectedness calculated for an event is greater than this threshold limit, the event is considered unexpected or anomalous.
Default: 0.01
labelonly
Datatype: labelonly=<bool>
Description: Specifies if you want the output result set to include all events or only the events that are above the threshold value. The unexpectedness field is appended to all events. If labelonly=true, no events are removed. If labelonly=false, events that have a unexpectedness score less than the threshold are removed from the output result set.
Default: false
normalize
Datatype: normalize=<bool>
Description: Specifies whether or not to normalize numeric text in the fields. All characters in the field from 0 to 9 are considered identical for purposes of the algorithm. The placement and quantity of the numbers remains significant. When a field contains numeric data that should not be normalized but treated as categories, set normalize=false.
Default: true
maxvalues
Datatype: maxvalues=<int>
Description: Specifies the size of the sliding set of previous events to include when determining the unexpectedness of a field value. By default the calculation uses the previous 100 events for the comparison. If the current event number is 1000, the calculation uses the values in events 900 to 999 in the calculation. If the current event number is 1500, the calculation uses the values in events 1400 to 1499 in the calculation. You can specify a number between 10 and 10000. Increasing the value of maxvalues increases the total CPU cost per event linearly. Large values have very long search runtimes.
Default: 100
field
Datatype: field=<field>
Description: The field to analyze when determining the unexpectedness of an event.
Default: _raw
blacklist
Datatype: blacklist=<filename>
Description: The name of a CSV file that contains a list of events that are expected and should be ignored. Any incoming event that is similar to an event in the blacklist is treated as not anomalous, or expected, and given an unexpectedness score of 0.0. The CSV file must be located in the $SPLUNK_HOME/var/run/splunk/ directory on the search head. If you have Splunk Cloud and want to configure a blacklist file, file a Support ticket.
blacklistthreshold
Datatype: blacklistthreshold=<num>
Description: Specifies a similarity score threshold for matching incoming events to blacklisted events. If the incoming event has a similarity score above the blacklistthreshold, the event is marked as unexpected.
Default: 0.05
by-clause
Syntax: by <fieldlist>
Description: Use to specify a list of fields to segregate the results for anomaly detection. For each combination of values for the specified fields, the events with those values are treated entirely separately.

Examples

1. Specify a blacklist file of the events to ignore

The following example shows the interesting events, ignoring any events in the blacklist 'boring events'. Sort the event list in descending order, with highest value in the unexpectedness field listed first.

... | anomalies blacklist=boringevents | sort -unexpectedness

2. Find anomalies in transactions

This example uses transactions to find regions of time that look unusual.

... | transaction maxpause=2s | anomalies

3. Identify anomalies by source

Look for anomalies in each source separately. A pattern in one source does not affect that it is anomalous in another source.

... | anomalies by source

4. Specify a threshold when identifying anomalies

This example shows how to tune a search for anomalies using the threshold value. Start with a search that uses the default threshold value.

index=_internal | anomalies BY group | search group=*

This search looks at events in the _internal index and calculates an unexpectedness score for sets of events that have the same group value.

  • The sliding set of events that are used to calculate the unexpectedness score for each unique group value includes only the events that have the same group value.
  • The search command is used to show events that only include the group field.

The unexpectedness and group fields appear in the list of Interesting fields. Click on the field name and then click Yes to move the field to the Selected fields list. The fields are moved and also appear in the search results. Your results should look something like the following image.

This image shows the search results. The unexpectedness and group fields are in the Selected fields list and are highlighted in the search results.

The key-value pairs in the first event include group=pipeline, name=indexerpipe, processor=indexer, cpu_seconds=0.022, and so forth.

With the default threshold, which is 0.01, you can see that some of these events might be very similar. The next search increases the threshold a little:

index=_internal | anomalies threshold=0.03 by group | search group=*

This image shows the search results with the new threshold.

With the higher threshold value, the timestamps and key-value pairs show more distinction between each of the events.

Also, you might not want to hide the events that are not anomalous. Instead, you can add another field to your events that tells you whether or not the event is interesting to you. One way to do this is with the eval command:

index=_internal | anomalies threshold=0.03 labelonly=true by group | search group=* | eval threshold=0.03 | eval score=if(unexpectedness>=threshold, "anomalous", "boring")

This search uses labelonly=true so that the boring events are still retained in the results list. The eval command is used to define a field named threshold and set it to the threshold value. This has to be done explicitly because the threshold attribute of the anomalies command is not a field.

The second eval command is used to define another new field, score, that is either "anomalous" or "boring" based on how the unexpectedness compares to the threshold value. The following image shows a snapshot of the results.

This image shows the score field added to the Selected field list. The score values are highlighted in the search results.

See also

anomalousvalue, cluster, kmeans, outlier

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has using the anomalies command.

PREVIOUS
analyzefields
  NEXT
anomalousvalue

This documentation applies to the following versions of Splunk Cloud: 7.2.4, 7.1.3, 7.1.6, 7.2.3


Comments

Mykol j
There are 2 thresholds in the syntax for this command. One is "threshold" and the other is "blacklistthreshold". The examples only show "threshold", which is the very first argument in the syntax.

Lstewart splunk, Splunker
February 11, 2019

Note: the argument described in the syntax section "blacklistthreshold" is really "threshold". it's correctly shown in the examples...

Mykol j
February 7, 2019

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters