Splunk® Enterprise

Alerting Manual

Acrobat logo Download manual as PDF


Splunk Enterprise version 8.2 is no longer supported as of September 30, 2023. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
Acrobat logo Download topic as PDF

Alert type and triggering scenarios

Once you choose a scheduled or a real-time alert, you can configure how results trigger the alert. Depending on the events that you are monitoring, you might need a real-time alert that triggers with every result or a scheduled alert that only triggers if results meet certain conditions. The following scenarios show different use cases for alert types and triggering.

Scheduled alert

Use a scheduled alert to search for events on a regular basis and monitor whether they meet specific conditions. A scheduled alert is useful if immediate or real-time monitoring is not a priority.

Scenarios

  • An online retailer has a daily goal of 500 sales. An admin for the retailer creates a scheduled alert to monitor sales performance. The admin schedules the alert to search for sales events each day at 23:00. She configures the alert to trigger if the number of results is lower than 500.
  • An admin wants to monitor how frequently users follow a bad link to the 404 error page. The admin creates a scheduled alert that searches for 404 errors every hour and triggers if there are more than 100 results.
  • An admin creates a scheduled alert to check if a particular host has not sent any data to the Splunk platform in the last few hours. He schedules the alert to search for events from the host every three hours. The admin configures the alert to trigger if there are no search results.

Real-time alert

Real-time alerts search for events continuously. They can be useful in situations where immediate monitoring and responses are important. You can use real-time alerts that trigger once per result or only if certain conditions are met within a specific rolling time window.

Per-result triggering

A real-time alert with a per-result triggering condition is sometimes known as a "per-result alert". Use this alert type and triggering to search continuously for events and to receive notifications when events occur.

Caution: In a high availability deployment, use per-result triggering with caution. If a peer is not available, a real-time search does not warn that the search might be incomplete. It is recommended to use a scheduled alert for this deployment.

Scenarios

Here are some examples of using a real-time alert with per-result triggering.

  • A social networking website admin wants to know whenever login failures occur. She sets up a real-time alert to search for failed login attempts. She chooses a per-result trigger condition so that she can track every failed login attempt.
  • An admin wants to monitor a set of hosts for errors in real-time. Some errors need a more urgent response than others. The admin sets up a real-time alert with a per-result trigger condition. He throttles the alert using a field that represents the less urgent error code and a one hour suppression period. The alert triggers for every urgent error, but at most once in an hour for less urgent errors.


Rolling time window triggering

A real-time alert with rolling time window triggering is sometimes known as a "rolling window alert". This alert type and triggering are useful when a specific time window is an important part of the event pattern you are monitoring in real time.

Scenarios

Here are some examples of using a real-time alert with rolling time window triggering.

  • An admin wants a notification whenever a user has three failed logins within a ten minute period. The admin sets up a real-time alert to search for failed logins and configures a rolling ten minute time window. The admin throttles the alert so that it triggers only once in an hour for failed logins from the same user.
  • An admin wants to know whenever a web application has more than five database connection errors in a minute. She configures a real-time alert to search for error events and specifies a one minute rolling window. If the search returns one result and then four more results five minutes later, the alert does not trigger. However, the alert triggers if the search returns five results within a one minute span.

Additional resources

See Throttle alerts to review additional throttling scenarios.

Last modified on 12 February, 2016
PREVIOUS
Alert types
  NEXT
Create scheduled alerts

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.5, 8.0.10, 7.2.1, 7.0.1, 8.0.4, 8.0.9, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.2.0, 8.0.6, 8.0.7, 8.0.8


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters