Splunk Cloud Platform

Alerting Manual

Throttle alerts

Use throttling to suppress alert triggering for a specific time period. Alerts can trigger frequently because of similar search results or scheduling.

Throttling an alert is different from configuring alert trigger conditions. Trigger conditions evaluate an alert's initial search results to check for specified field counts, event timing, or other patterns. To review alert triggering information, see Configuring alert trigger conditions.

Throttle configuration

When creating or editing an alert, you can enable and configure alert throttling, also known as suppression.

Alert type Triggering option How to configure throttling
Scheduled Once Indicate a suppression period using the time value field and dropdown increments. Time values must be greater than zero.
Scheduled Per-result
  1. Type one or more comma-separated fields to check for matching values in events. Events with the same value for these fields are suppressed.

    As an example, you might configure suppression on the field product_category and the field best_sellers. After an alert on one event where both the product_category field AND the best_sellers have value arcade, subsequent events with the arcade value in the product_category and the best_sellers field are suppressed during the throttling time period.
  2. Indicate a suppression period using the time value field and dropdown increments. Time values must be greater than zero.
Real-time Rolling time window Indicate a suppression period using the time value field and dropdown increments. Time values must be greater than zero.
Real-time Per-result
  1. Type one or more comma-separated fields to check for matching values in events. Events with the same value for these fields are suppressed.

    As an example, you might configure suppression on a product_category field. After an alert on one event with the product_category value arcade, subsequent events with the arcade value in the product_category field are suppressed during the throttling time period.
  2. Indicate a suppression period using the time value field and dropdown increments. Time values must be greater than zero.

If you have throttling set for an existing alert action, editing the details of the alert or the throttle configuration causes the throttling to be disregarded. This includes any changes to fields you throttle on, the SPL in the correlation search, the cron schedule, and so on. The change causes the throttle file, which notes how long to ignore events, to be removed. Therefore the throttling does not occur until the next event is triggered based on the new parameters.

Throttling scenarios

  • An admin uses a real-time alerts with per-result triggering to monitor system events, including errors. System events occur twenty or more times per minute. For notification purposes, alert triggers can be suppressed for an hour. The admin uses field values and a one hour suppression period to throttle the events.
  • A real-time alert with per-result triggering monitors disk errors. Some events in the alert's search results have the same clientip or host values but can cause multiple alert triggers in a short amount of time. An admin throttles the alert so that, after an initial alert triggers, subsequent triggering is suppressed for ten minutes.
  • A scheduled alert searches for sales events on an hourly basis. The alert triggers whenever the number of results rises by 100 and is configured to send an email notification to the sales team. The sales team wants to limit email notifications. An admin throttles the alert so that triggering is suppressed for three hours after an initial alert triggers and initializes an email notification.

If you have a set of alerts that run over similar datasets, they can each alert on the same data at the same time. This means that their collective notification frequency can still be high, even when you have throttling rules set up for all of them. You can create suppression groups for these alerts so that when one of them alerts, the entire group of alerts is suppressed.

See Define alert suppression groups to throttle sets of similar alerts.

Throttle scheduled and real-time searches

Throttling for alerting works similarly to throttling for scheduled and real-time searches.

If you have scheduled searches that run frequently and you do not want to be notified each time results generate, set the throttling controls to suppress the alert for a longer time period.

For real-time searches, if you configure an alert so that it triggers once when a specific triggering condition is met, you do not need to configure throttling. If the alert triggers for each result, you might need to configure throttling to suppress additional alerts.

When you configure throttling for a real-time search, start with a throttling period that matches the length of the base search time range. Expand the throttling period if necessary. This prevents multiple notifications for a given event.

Last modified on 04 June, 2021
Configure alert trigger conditions   Define alert suppression groups to throttle sets of similar alerts

This documentation applies to the following versions of Splunk Cloud Platform: 9.3.2408, 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release)


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters