Splunk® Enterprise

Alerting Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Set up alert actions

You can enable the following alert actions:

  • Send email notification.
    The email notification can include information related to the alert.
  • Run scripts.
  • Enable RSS notification for the alert.
  • Enable summary indexing for alerts.
  • Track the alert in Splunk Enterprise Settings.

Email notification

You can configure an alert to send an email notification to specified recipients when the alert triggers. The email notification is a multipart MIME message that includes both HTML and text parts.

You configure the email notification action for an alert when you save the alert from the Search page. You can also configure email notification from the Alerts Page and directly from a search command.

Before you can send an email notification, configure the email notification settings in Settings. See Configure email notification settings.

Email notification contexts

There are several contexts from which you can send email notifications. The email options available differ, depending on the context.

  • Alert actions
    Send email notifications as an alert action from a search. Specify the notification from the Search Page, a listing in the Alerts Page, or directly from the search command.
  • Scheduled report
    Configure email notifications for a scheduled report either from a listing in the Reports Page or from a report.
  • Scheduled PDF delivery of dashboards
    Configure PDF delivery either from a listing in the Dashboards Page or from a dashboard.

This topic covers alert actions from a search job. See Schedule reports and Generate Dashboard PDFs for information on the other contexts for email notification.

Configure email notification for alerts

You configure email notifications from the Search Page when you save a search. You can also configure email notifications for an alert listed on the Alerts Page by editing an alert's actions. The procedure is the same as from the Search page.

After running a search from the Search page, save the search as an alert and configure email notification settings.

  1. Run the search.
  2. Select Save As > Alert.
  3. Provide a Title and other information about the alert. Click Next.
  4. Select Send Email.
    The Email Actions dialog box opens.
  5. Specify the following:

    • To, CC, and BCC email recipients.
      Specify a comma-separated list of email recipients.
    • Priority
      Enforcement of priority depends on your email client.
    • Subject
    • Message
    • Include
      You can include the following items:

      Information about the search
        Link to the alert
        Search string
        Trigger condition
        Trigger time

      Information about search results
        Link to results
        Inline listing of results, as a table, raw events, or CSV file
        Results as a PDF attachment
        Results as a CSV attachment
  6. Specify other alert actions.
    See Run a script and Create an RSS feed.
  7. Click Save.

Send email notification from a search command

You can send email notifications directly from the sendemail search command. For example:

index=main | head 5 | sendemail to=example@splunk.com server=mail.example.com subject="Here is an email notification" message="This is an example message" sendresults=true inline=true format=raw sendpdf=true

See the sendemail command listing in the Search Reference for details.

Use tokens in email notifications

A token is a type of variable that represents data generated by a search job. Splunk Enterprise provides tokens that you can use to include information generated by a search in the fields of an email:

  • To
  • Cc
  • Bcc
  • Subject
  • Message
  • Footer

Access the value of a token with the following syntax:


For example, place the following token in the subject field of an email notification to reference the search ID of a search job.

Search results from $job.sid$

Tokens available for email notifications

This section lists common tokens you can use in email notifications. There are four categories of tokens that access data generated from a search. The context for using the tokens differ.

Category Description Context
Search metadata Information about the search. Alert actions from search
Scheduled reports
Scheduled PDF delivery of dashboards
Search results Access results of a search Alert actions from search
Scheduled reports
Job information Data specific to a search job Alert actions from search
Scheduled reports
Server information Information about the Splunk Enterprise server Alert actions from search
Scheduled reports
Scheduled PDF delivery of dashboards

In addition to the common tokens listed in this topic, the savedsearches.conf and alert_action.conf configuration files list attributes whose values are available from tokens. To access these attribute values, place the attribute between the '$' token delimiters. For example, to access the subject of an email notification, reference the following attribute listed in savedsearches.conf:


Tokens that access search metadata

Common tokens that access information about a search. These tokens are available from the following contexts:

  • Alert actions
  • Scheduled reports
  • Scheduled PDF delivery of dashboards

Here are some of the common tokens available.

Token Description
$action.email.hostname$ Hostname of the email server.
$action.email.priority$ Priority of the search.
$app$ Name of the app containing the search.
$cron_schedule$ Cron schedule for the app.
$description$ Description of the search.
$name$ Name of the search.
$next_scheduled_time$ The next time the search runs.
$owner$ Owner of the search.
$results_link$ (Alert actions and scheduled reports only) Link to the search results.
$search$ The actual search.
$trigger_date$ (Alert actions only) The date that triggers the alert.
$trigger_time$ (Alert actions only) The scheduled time the alert runs.
$type$ Indicates if the search is from an alert, report, view, or the search command.
$view_link$ Link to view the saved report.
$alert.severity$ Severity level of the alert.
$alert.expires$ Time the alert expires.

Tokens available from results

From results, you use the result.<fieldname> token to access the first value of a specified field in search results. This token is available from the following contexts:

  • Alert actions
  • Scheduled reports
Token Description
$result.fieldname$ Returns the first value for the specified field name from the first result in the search. The field name must be present in the search.

Tokens that access job information

Common tokens that access data specific to a search job, such as the search ID or messages generated by the search job. These tokens are available from the following contexts:

  • Alert actions
  • Scheduled reports
Token Description
$job.earliestTime$ Initial time a search job starts.
$job.eventSearch$ Subset of the search that contains the part of the search before any transforming commands.
$job.latestTime$ Latest time recorded for the search job.
$job.messages$ List of error and debug messages generated by the search job.
$job.resultCount$ Number of results returned by the search job.
$job.runDuration$ Time, in seconds, that the search took to complete.
$job.sid$ Search ID.
$job.label$ Name given to the search job.

Tokens available from server

Common tokens that provide details available from your Splunk Enterprise server. They are available in the following contexts:

  • Alert actions
  • Scheduled reports
  • Scheduled PDF delivery of dashboards
Token Description
$server.build$ Build number of the Splunk Enterprise instance.
$server.serverName$ Server name hosting the Splunk Enterprise instance.
$server.version$ Version number of the Splunk Enterprise instance.

Deprecated email notification tokens

The following tokens from prior releases of Splunk Enterprise are deprecated.

Token Description
$results.count$ (Deprecated) Use $job.resultCount$.
$results.url$ (Deprecated) Use $results_link$.
$results.file$ (Deprecated) No equivalent available.
$search_id$ (Deprecated) Use $job.id$.

Configure email notification settings

Before you send an email notification for an alert, configure the email notification settings. Configure email notifications by editing the alert_actions.conf configuration file or from Splunk Web.

To configure email alert settings from a configuration file, see alert_actions.conf.

Configure email alert settings from Splunk Web.

  1. From Splunk Web, select Settings > System settings > Email settings.
  2. Select Mail Server Settings:

    • Mail host
      The default is localhost. To schedule PDF delivery requires additional configuration of user roles. See User role configuration to schedule PDF delivery of dashboards.
    • Email security
    • Username
      User name and password are optional. You do not need to specify these fields to configure email notification.
  3. Specify Email Format:

    • Link hostname
      The host name of the server from which to create URLs for outgoing results.

      This is also the search head host name for the instance sending requests to a PDF Report Server. Use the Remote PDF Report Server to print dashboards built with advanced XML. Set this option only if your environment improperly auto-detects the host name. See Dashboards and forms that use advanced XML.
    • Send emails as
      Add an email or string to specify the sender.
    • Email footer
      Text to be added as a footer to each email. You can specify tokens in the email footer. See Use tokens in email notifications.
  4. Specify PDF Report Settings.

    • Report Paper Size
    • Report Paper Orientation
  5. Click Save.

User role configuration to schedule PDF delivery of dashboards

For a user to schedule PDF delivery of dashboards, the user role must contain the following capabilities:

  • schedule_search
  • admin_all_objects
    This capability is required only if the mail host requires log-in credentials.

See About defining roles with capabilities.

Run a script for an alert action

You can run an alert script when a alert triggers. Select Run a script under Enable actions. Enter the file name of the script that you want to run.

For example, you can configure an alert to run a script that generates a Simple Network Management Protocol (SNMP) trap notification. The script sends the notification to another system such as a Network Systems Management console. You can configure a different alert that runs a script that calls an API, which in turn sends the triggering event to another system.

Note: For security reasons, place all alert scripts in either of the following locations:

  • $SPLUNK_HOME/bin/scripts
  • $SPLUNK_HOME/etc/apps/<AppName>/bin/scripts

For details on alert script configuration using savedsearches.conf with a shell script or batch file that you create, see "Configure scripted alerts" in this manual.

If you are having trouble with alert scripts, see Troubleshooting alert scripts on the Splunk Community Wiki.

Show triggered alerts in the Alert manager

Select the List in Triggered Alerts action to display triggered alerts in the Alert manager. The Alert manager lists details of triggered alerts for 24 hours or a specified duration. See "Review triggered alerts" in this manual.

Give tracked alerts a severity level

When listing a triggered alert, you can specify a Severity level. Severity levels are informational only. They let you group and highlight alerts in the Alert Manager according to the severity levels. You decide which level applies to the alert.

You can choose from the following severity levels. The default level is Medium.

  • Info
  • Low
  • Medium
  • High
  • Critical

Create an RSS feed

  • Note: The RSS feed alert action is not currently supported for search head clusters.

You can add an RSS feed for alert notifications. When the alert triggers, the alert generates notification to the RSS feed. An alert must trigger at least once to generate the RSS feed.

This alert action is available only from Settings.

  1. Go to Settings > Searches, reports, and alerts.
  2. Select the alert you are updating.
  3. Scroll to Alert actions.
  4. For Add to RSS, select Enable.
  5. Return to Settings > Searches, reports, and alerts.
  6. Click the RSS feed icon to subscribe to the feed.

    Alert rss.png
    You are given several options to subscribe to the feed.

When an alert with the Add to RSS action triggers, it generates a notification to its RSS feed. The feed is located at:


For example, here is the location for an RSS feed for an alert named "Errors in the last 24 hours", on a Splunk Enterprise instance using port 8000, and on a machine named "MyHost."


In Settings > Searches, reports, and alerts, click the RSS Feed icon to subscribe to the RSS feed.

Caution: The RSS feed is available to any user with access to the web server that displays the feed. Unauthorized users cannot follow the RSS link back to the Splunk Enterprise application to view the results of a specific search. But unauthorized users can see the summarization displayed in the RSS feed. The summarization includes the name of the search that was run and the number of results returned by the search.

This example shows the XML that generates the feed.

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
        <title>Alert: errors last15</title>
        <description>Reports Feed for report errors last15</description>
            <title>errors last15</title>
              Alert trigger: errors last15, results.count=123
            <pubDate>Mon, 01 Feb 2010 12:55:09 -0800</pubDate>

Specify fields to show in alerts through search language

The results of an alerting search job (in an alert email, for example) includes all the fields in those results. To include or exclude specific fields from the results, use the fields command in the base search for the alert.

  • To eliminate a field from the search results, pipe your search to fields - $FIELDNAME.
  • To add a field to the search results, pipe your search to fields + $FIELDNAME.

You can specify multiple fields in one string. The following search generates an alert that excludes $FIELD1 and $FIELD2, but includes $FIELD3 and $FIELD4.

yoursearch | fields - $FIELD1,$FIELD2 + $FIELD3,$FIELD4

Enable summary indexing in Settings

Summary indexing is an action that you can configure for any alert using Settings > Searches and Reports. Use summary indexing to perform analysis/reports on large amounts of data over long timespans. Typically this can be quite time consuming and a drain on performance if several users are running similar searches on a regular basis.

Caution: For summary indexing you typically use reporting commands to properly construct the search that populates the summary index. Before setting up a summary index, read "Use summary indexing for increased reporting efficiency" in the Knowledge Manager manual.

With summary indexing, you base an alert on a search that computes sufficient statistics (a summary) for events covering a slice of time. The search is set up so that each time it runs on its schedule, the search results are saved into a summary index that you designate. You can then run searches against this smaller (and thus faster) summary index instead of working with the much larger dataset from which the summary index receives its events.

Note: You do not need to use summary indexing for searches that already benefit from report acceleration. For more information and a distinction between these two methods of speeding up slow running searches, see "About report acceleration and summary indexing" in the Knowledge Manager manual.

To set up summary indexing for an alert, go to Settings > Searches and Reports, and either add a new report or open up the detail page for an existing search or alert. (You cannot set up summary indexing through the Create Alert window.) To enable the summary index to gather data on a regular interval, set its Alert condition to always and then select Enable under Summary indexing at the bottom of the view.

Update and expand alert functionality
Alert examples

This documentation applies to the following versions of Splunk® Enterprise: 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


In response to 0range, the preferred method to edit a search string for an alert is the following:
1. Go the Alerts page.
2. Select Open in Search for the alert you want to modify.
3. Modify the Search.
4. Run the Search.
5. Select Save.

To edit email actions:
1. Select the Alert from Alerts page.
2. For Actions, click Edit.
3. Click Send Email and modify the email actions.

February 12, 2015

So we do have to change the search string on one page and the email action on another?

May 30, 2014

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters