Splunk® Enterprise

Alerting Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Throttle alerts and related searches

Use throttling to limit alert frequency

Use throttling to reduce the frequency at which an alert triggers. An alert can trigger frequently based on similar results that the search returns. The schedule to run an alert can also cause the alert to trigger frequently. To reduce the frequency of the alert firing, configure the following:

  • A time period in which to suppress results.
  • Field values that the search returns.

For example, you can create an alert that fires when a system error occurs. For this example, assume that when the system error occurs, it occurs 20 or more times each minute. However, you want to send an alert only once every hour. To reduce the frequency of the alert firing, configure throttling for the alert.

  1. From the Search Page, enter the following search:

    index=_internal log_level=ERROR

  2. Select Save As > Alert
  3. For Result Type, click Real Time to configure a per-result alert.
  4. Click Next.
  5. Select the actions you want to enable.
  6. Select Throttle.
  7. Enter log_level to suppress the alert for the field log_level.
    You can configure throttling to suppress on more than one field. Use a comma-delimited list to specify fields for throttling.
  8. Enter 1 hour as the time to suppress triggering for the alert.

    Alert throttle.png
  9. Click Save.

You can set up a per-result alert that throttles events that share the same clientip and host values. For example, a real-time search with a 60 second window triggers an alert every time an event with disk error appears. Ten events with the error message that occurs in the window triggers five disk error alerts, which is ten alerts within one minute. If the alert sends an email notification each time it triggers, you can overwhelm a email Inbox.

You can configure throttling so that when one alert of this type triggers, it suppresses all successive alerts of the same type for the next 10 minutes. After each successive 10 minutes period pass, the alert can trigger again.

Throttle scheduled and real-time searches

If you have scheduled searches that run frequently and you do not want to be notified for each run, set the throttling controls to suppress the alert to a longer time window.

For real-time searches, if you configure an alert so that it fires once for a trigger condition, you do not need to configure throttling. If the alert fires more than once for the trigger condition, consider throttling to suppress results.

When you configure throttling for a real-time search, start with a throttling period that matches the length of the base search's time window. Expand the throttling period if necessary. This prevents multiple notifications for a given event.

PREVIOUS
Create rolling-window alerts
  NEXT
Set up alert actions

This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters