Splunk® Enterprise

Alerting Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Review triggered alerts

You can see records of your recently triggered alerts in the Alert Manager. The Alert Manager displays records of triggered alerts that have List in Triggered Alerts enabled as an alert action in the alert actions of the Save As alert dialog:

60 saveasalert page2 enableactions listintrigger sendemail.png

Alternatively, if you are creating or updating an alert in Settings > Searches and Reports, you can enable the List in Triggered Alerts alert action to have that alert's triggered alert records appear in the Alert Manager:

60 managealert listintriggeredevents wastracking.png

Open the Alert Manager by clicking the Alerts link at the upper right-hand corner of the Splunk UI. It opens in a separate window.

60 alertmgr triggeredalerts.png

Note: The Alert Manager displays records for triggered alerts that are based on existing reports. They will continue to appear even if you disable the alerting aspect of those searches after the alerts were triggered. The Alert Manager will not display records of triggered alerts that are based on deleted reports, however.

For more information about alerts and alert definition, see "About alerts" in this manual.

Setting alert expiration

Triggered alert records are designed to expire (be automatically deleted from the Alert Manager) after a set period of time. You define triggered alert record expiration periods at the individual alert level.

For example, say you have a Firewall breach alert with an Expiration setting of 2 days. If the Firewall breach alert is triggered at 3pm, the related alert record will be deleted from the Alert Manager at 3pm the two days later.

When an alert is first created it has a default expiration time of 24 hours. This means that the triggered alert records for a new alert will disappear a day after it appears in the Alert Manager unless you give it a different expiration period.

60 managealert expiration.png

To change the default expiration period for an alert's triggered alert records, go to the detail page for the base report in Manager > Searches and Reports and set Expiration to your desired number of seconds, minutes, hours, or days (or just select one of the pre-defined time ranges from the list).

Using the Alert Manager

You can filter the Alert Manager listing by app, alert severity, and alert type. You can also search for specific keywords using the search box. The keyword search applies to fired alert names (which are the same as the names of the searches or reports upon which the alerts are based) and the alert severity (so you can search specifically for alerts of Critical severity, if necessary).

Additionally, the Alert Manager enables you to manually delete individual alert records.

Note that the Severity column enables you to quickly spot those alert records that have been given a higher severity level (such as High or Critical). When you define or update your alert definition, use the Severity field to set the alert severity level. The severity label is for informational purposes only; there is no additional functionality associated with it.

The Type column indicates whether the alert is running in Real-time or is Scheduled to run on a regular interval. The Mode column indicates whether the alert represents a set of events (Digest) or a single event (Per Result).

Click View results for a specific alert record to see the results captured by that alert in another browser tab. This is a search job artifact; it won't contain any events that weren't returned by the search job that originally triggered the alert.

Click Edit search for a specific alert record to edit the underlying search for the alert. You can change the search string and/or redefine the alert definition.

Setting up Alert Manager tracking when upgrading from a pre-4.2 Splunk version

If you're upgrading from a 4.1.x version of Splunk, be aware that by default existing alerts do NOT show up in the Alert Manager. To quickly update your existing alerts so that they show up in the Alert Manager, edit the relevant copy of savedsearches.conf. Add alert.track = true to the stanzas of each report that you have set up as an alert and want to see tracked in the Alert Manager. Review "About configuration files" in the Admin Manual for details about configuration files.

Alert examples
Configure alerts in savedsearches.conf

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters