Splunk Cloud Platform

Alerting Manual

Alert scheduling tips

This topic presents best practices and suggestions for working with scheduled alerts.

Best practices

Coordinate an alert schedule and search time range

Coordinating an alert schedule with the search time range prevents event data from being evaluated twice by the search. If search time range exceeds the search schedule, event data sets can overlap.

When a search time range is shorter than the time range for the scheduled alert, an event might never be evaluated.

Schedule alerts with at least one minute of delay

This practice is important in distributed search deployments where event data might not reach the indexer immediately. A delay ensures that you are counting all events, not just the events that were indexed first.

Best practices example

This example shows how to configure an alert that builds 30 minutes of delay into the alert schedule. Both the search time range and the alert schedule span one hour, so there are no event data overlaps or gaps.

  1. From the Search Page, create a search and select Save As > Alert.
  2. In the Save As Alert dialog, specify the following options as shown.
    • Title: Alert Example (30 Minute Delay)
    • Alert Type: Scheduled
    • Time Range: Run on Cron Schedule
    • Earliest: -90m
    • Latest: -30m
    • Cron Expression: 30 * * * *
  3. Continue defining actions for the alert.

Earliest and Latest values set the search time range from 90 minutes before the search launches to 30 minutes before the search launches. The alert runs runs hourly at 30 minutes past the hour. It collects event data from a one hour period. When the scheduled search begins at a designated time, such as 3:30 p.m., it collects the event data indexed from 2:00 pm to 3:00 pm.

Manage concurrent scheduled search priority

Depending on your deployment, you might be able to run only one scheduled search at a time. In this case, even if you schedule multiple searches to run at the same time, the search scheduler ensures that scheduled searches run consecutively.

You might need to change scheduled search priority to ensure that a search obtains current data or to prevent gaps in data collection. If you have Splunk Enterprise, you can configure scheduled search priority by editing the savedsearches.conf configuration file. See Configure the priority of scheduled reports in the Reporting Manual for more information.

Differences between scheduled reports and alerts

A scheduled report is like a scheduled or real-time alert in certain ways. You can schedule a report and set up an action to run each time the scheduled report runs.

Scheduled reports are different from alerts, however, because a scheduled report's action runs every time the report is run. The report action does not depend on trigger conditions.

As an example, you can monitor guest check-ins at a hotel using an hourly search. Here are the differences between a scheduled report and a scheduled alert with email notification actions.

  • Scheduled report: runs its action and sends an email every time the report completes, even if there are no search results showing check-ins. In this case, you get an email notification every hour.
  • Scheduled alert: only runs alert action when it is triggered by search results showing one or more check-in events. In this case, you only get an email notification if results trigger the alert action.

For more information, see Schedule reports in the Reporting Manual.

Last modified on 03 October, 2016
Use cron expressions for alert scheduling   Create real-time alerts

This documentation applies to the following versions of Splunk Cloud Platform: 8.2.2112, 8.2.2201, 8.2.2202, 9.0.2205, 8.2.2203, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308 (latest FedRAMP release), 9.1.2312, 9.2.2403

Was this topic useful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters