
Set up alert actions
You can enable the following alert actions:
- Send email notification.
The email notification can include information related to the alert. - Run scripts.
- Enable RSS notification for the alert.
- Enable summary indexing for alerts.
- Track the alert in Splunk Enterprise Settings.
Email notification
You can configure an alert to send an email notification to specified recipients when the alert triggers. The email notification is a multipart MIME message that includes both HTML and text parts.
You configure the email notification action for an alert when you save the alert from the Search page. You can also configure email notification from the Alerts Page and directly from a search command.
Before you can send an email notification, configure the email notification settings in Settings. See Configure email notification settings.
Email notification contexts
There are several contexts from which you can send email notifications. The email options available differ, depending on the context.
- Alert actions
Send email notifications as an alert action from a search. Specify the notification from the Search Page, a listing in the Alerts Page, or directly from the search command.
- Scheduled report
Configure email notifications for a scheduled report either from a listing in the Reports Page or from a report.
- Scheduled PDF delivery of dashboards
Configure PDF delivery either from a listing in the Dashboards Page or from a dashboard.
This topic covers alert actions from a search job. See Schedule reports and Generate Dashboard PDFs for information on the other contexts for email notification.
Configure email notification for alerts
You configure email notifications from the Search Page when you save a search. You can also configure email notifications for an alert listed on the Alerts Page by editing an alert's actions. The procedure is the same as from the Search page.
After running a search from the Search page, save the search as an alert and configure email notification settings.
- Run the search.
- Select Save As > Alert.
- Provide a Title and other information about the alert. Click Next.
- Select Send Email.
The Email Actions dialog box opens. - Specify the following:
- To, CC, and BCC email recipients.
Specify a comma-separated list of email recipients. - Priority
Enforcement of priority depends on your email client. - Subject
- Message
- Include
You can include the following items:
Information about the search
Link to the alert
Search string
Trigger condition
Trigger time
Information about search results
Link to results
Inline listing of results, as a table, raw events, or CSV file
Results as a PDF attachment
Results as a CSV attachment
- To, CC, and BCC email recipients.
- Specify other alert actions.
See Run a script and Create an RSS feed. - Click Save.
Send email notification from a search command
You can send email notifications directly from the sendemail
search command. For example:
index=main | head 5 | sendemail to=example@splunk.com server=mail.example.com subject="Here is an email notification" message="This is an example message" sendresults=true inline=true format=raw sendpdf=true
See the sendemail command listing in the Search Reference for details.
Use tokens in email notifications
A token is a type of variable that represents data generated by a search job. Splunk Enterprise provides tokens that you can use to include information generated by a search in the fields of an email:
- To
- Cc
- Bcc
- Subject
- Message
- Footer
Access the value of a token with the following syntax:
For example, place the following token in the subject field of an email notification to reference the search ID of a search job.
Tokens available for email notifications
This section lists common tokens you can use in email notifications. There are four categories of tokens that access data generated from a search. The context for using the tokens differ.
Category | Description | Context |
---|---|---|
Search metadata | Information about the search. | Alert actions from search Scheduled reports Scheduled PDF delivery of dashboards |
Search results | Access results of a search | Alert actions from search Scheduled reports |
Job information | Data specific to a search job | Alert actions from search Scheduled reports |
Server information | Information about the Splunk Enterprise server | Alert actions from search Scheduled reports Scheduled PDF delivery of dashboards |
In addition to the common tokens listed in this topic, the savedsearches.conf
and alert_action.conf
configuration files list attributes whose values are available from tokens. To access these attribute values, place the attribute between the '$' token delimiters. For example, to access the subject of an email notification, reference the following attribute listed in savedsearches.conf
:
Tokens that access search metadata
Common tokens that access information about a search. These tokens are available from the following contexts:
- Alert actions
- Scheduled reports
- Scheduled PDF delivery of dashboards
Here are some of the common tokens available.
Token | Description |
---|---|
$action.email.hostname$ | Hostname of the email server. |
$action.email.priority$ | Priority of the search. |
$app$ | Name of the app containing the search. |
$cron_schedule$ | Cron schedule for the app. |
$description$ | Description of the search. |
$name$ | Name of the search. |
$next_scheduled_time$ | The next time the search runs. |
$owner$ | Owner of the search. |
$results_link$ | (Alert actions and scheduled reports only) Link to the search results. |
$search$ | The actual search. |
$trigger_date$ | (Alert actions only) The date that triggers the alert. |
$trigger_time$ | (Alert actions only) The scheduled time the alert runs. |
$type$ | Indicates if the search is from an alert, report, view, or the search command. |
$view_link$ | Link to view the saved report. |
$alert.severity$ | Severity level of the alert. |
$alert.expires$ | Time the alert expires. |
Tokens available from results
From results, you use the result.<fieldname>
token to access the first value of a specified field in search results. This token is available from the following contexts:
- Alert actions
- Scheduled reports
Token | Description |
---|---|
$result.fieldname$ | Returns the first value for the specified field name from the first result in the search. The field name must be present in the search. |
Tokens that access job information
Common tokens that access data specific to a search job, such as the search ID or messages generated by the search job. These tokens are available from the following contexts:
- Alert actions
- Scheduled reports
Token | Description |
---|---|
$job.earliestTime$ | Initial time a search job starts. |
$job.eventSearch$ | Subset of the search that contains the part of the search before any transforming commands. |
$job.latestTime$ | Latest time recorded for the search job. |
$job.messages$ | List of error and debug messages generated by the search job. |
$job.resultCount$ | Number of results returned by the search job. |
$job.runDuration$ | Time, in seconds, that the search took to complete. |
$job.sid$ | Search ID. |
$job.label$ | Name given to the search job. |
Tokens available from server
Common tokens that provide details available from your Splunk Enterprise server. They are available in the following contexts:
- Alert actions
- Scheduled reports
- Scheduled PDF delivery of dashboards
Token | Description |
---|---|
$server.build$ | Build number of the Splunk Enterprise instance. |
$server.serverName$ | Server name hosting the Splunk Enterprise instance. |
$server.version$ | Version number of the Splunk Enterprise instance. |
Deprecated email notification tokens
The following tokens from prior releases of Splunk Enterprise are deprecated.
Token | Description |
---|---|
$results.count$ | (Deprecated) Use $job.resultCount$. |
$results.url$ | (Deprecated) Use $results_link$. |
$results.file$ | (Deprecated) No equivalent available. |
$search_id$ | (Deprecated) Use $job.id$. |
Configure email notification settings
Before you send an email notification for an alert, configure the email notification settings. Configure email notifications by editing the alert_actions.conf
configuration file or from Splunk Web.
To configure email alert settings from a configuration file, see alert_actions.conf.
Configure email alert settings from Splunk Web.
- From Splunk Web, select Settings > System settings > Email settings.
- Select Mail Server Settings:
- Mail host
The default is localhost. To schedule PDF delivery requires additional configuration of user roles. See User role configuration to schedule PDF delivery of dashboards. - Email security
- Username
Password
User name and password are optional. You do not need to specify these fields to configure email notification.
- Mail host
- Specify Email Format:
- Link hostname
The host name of the server from which to create URLs for outgoing results.
This is also the search head host name for the instance sending requests to a PDF Report Server. Use the Remote PDF Report Server to print dashboards built with advanced XML. Set this option only if your environment improperly auto-detects the host name. See Dashboards and forms that use advanced XML. - Send emails as
Add an email or string to specify the sender. - Email footer
Text to be added as a footer to each email. You can specify tokens in the email footer. See Use tokens in email notifications.
- Link hostname
- Specify PDF Report Settings.
- Report Paper Size
- Report Paper Orientation
- Click Save.
User role configuration to schedule PDF delivery of dashboards
For a user to schedule PDF delivery of dashboards, the user role must contain the following capabilities:
- schedule_search
- admin_all_objects
This capability is required only if the mail host requires log-in credentials.
See About defining roles with capabilities.
Run a script for an alert action
You can run an alert script when a alert triggers. Select Run a script under Enable actions. Enter the file name of the script that you want to run.
For example, you can configure an alert to run a script that generates a Simple Network Management Protocol (SNMP) trap notification. The script sends the notification to another system such as a Network Systems Management console. You can configure a different alert that runs a script that calls an API, which in turn sends the triggering event to another system.
- Note: For security reasons, place all alert scripts in either of the following locations:
-
$SPLUNK_HOME/bin/scripts
$SPLUNK_HOME/etc/apps/<AppName>/bin/scripts
-
For details on alert script configuration using savedsearches.conf
with a shell script or batch file that you create, see "Configure scripted alerts" in this manual.
If you are having trouble with alert scripts, see Troubleshooting alert scripts on the Splunk Community Wiki.
Show triggered alerts in the Alert manager
Select the List in Triggered Alerts action to display triggered alerts in the Alert manager. The Alert manager lists details of triggered alerts for 24 hours or a specified duration. See "Review triggered alerts" in this manual.
Give tracked alerts a severity level
When listing a triggered alert, you can specify a Severity level. Severity levels are informational only. They let you group and highlight alerts in the Alert Manager according to the severity levels. You decide which level applies to the alert.
You can choose from the following severity levels. The default level is Medium.
- Info
- Low
- Medium
- High
- Critical
Create an RSS feed
- Note: The RSS feed alert action is not currently supported for search head clusters.
You can add an RSS feed for alert notifications. When the alert triggers, the alert generates notification to the RSS feed. An alert must trigger at least once to generate the RSS feed.
This alert action is available only from Settings.
- Go to Settings > Searches, reports, and alerts.
- Select the alert you are updating.
- Scroll to Alert actions.
- For Add to RSS, select Enable.
- Return to Settings > Searches, reports, and alerts.
- Click the RSS feed icon to subscribe to the feed.
You are given several options to subscribe to the feed.
When an alert with the Add to RSS action triggers, it generates a notification to its RSS feed. The feed is located at:
http://[splunkhost]:[port]/[locale]/rss/[alert_name]
For example, here is the location for an RSS feed for an alert named "Errors in the last 24 hours", on a Splunk Enterprise instance using port 8000, and on a machine named "MyHost."
http://MyHost:8000/en-US/rss/Errors%20in%20the%20last%2024%20hours
.
In Settings > Searches, reports, and alerts, click the RSS Feed icon to subscribe to the RSS feed.
- Caution: The RSS feed is available to any user with access to the web server that displays the feed. Unauthorized users cannot follow the RSS link back to the Splunk Enterprise application to view the results of a specific search. But unauthorized users can see the summarization displayed in the RSS feed. The summarization includes the name of the search that was run and the number of results returned by the search.
This example shows the XML that generates the feed.
<?xml version="1.0" encoding="UTF-8"?> <rss version="2.0"> <channel> <title>Alert: errors last15</title> <link> http://localhost:8000/app/search/@go?sid=scheduler_Z2d1cHRh </link> <description>Reports Feed for report errors last15</description> <item> <title>errors last15</title> <link> http://localhost:8000/app/search/@go?sid=scheduler_Z2d1cHRh </link> <description> Alert trigger: errors last15, results.count=123 </description> <pubDate>Mon, 01 Feb 2010 12:55:09 -0800</pubDate> </item> </channel> </rss>
Specify fields to show in alerts through search language
The results of an alerting search job (in an alert email, for example) includes all the fields in those results. To include or exclude specific fields from the results, use the fields
command in the base search for the alert.
- To eliminate a field from the search results, pipe your search to
fields - $FIELDNAME
. - To add a field to the search results, pipe your search to
fields + $FIELDNAME
.
You can specify multiple fields in one string. The following search generates an alert that excludes $FIELD1
and $FIELD2
, but includes $FIELD3
and $FIELD4
.
yoursearch | fields - $FIELD1,$FIELD2 + $FIELD3,$FIELD4
Enable summary indexing in Settings
Summary indexing is an action that you can configure for any alert using Settings > Searches and Reports. Use summary indexing to perform analysis/reports on large amounts of data over long timespans. Typically this can be quite time consuming and a drain on performance if several users are running similar searches on a regular basis.
- Caution: For summary indexing you typically use reporting commands to properly construct the search that populates the summary index. Before setting up a summary index, read "Use summary indexing for increased reporting efficiency" in the Knowledge Manager manual.
With summary indexing, you base an alert on a search that computes sufficient statistics (a summary) for events covering a slice of time. The search is set up so that each time it runs on its schedule, the search results are saved into a summary index that you designate. You can then run searches against this smaller (and thus faster) summary index instead of working with the much larger dataset from which the summary index receives its events.
- Note: You do not need to use summary indexing for searches that already benefit from report acceleration. For more information and a distinction between these two methods of speeding up slow running searches, see "About report acceleration and summary indexing" in the Knowledge Manager manual.
To set up summary indexing for an alert, go to Settings > Searches and Reports, and either add a new report or open up the detail page for an existing search or alert. (You cannot set up summary indexing through the Create Alert window.) To enable the summary index to gather data on a regular interval, set its Alert condition to always and then select Enable under Summary indexing at the bottom of the view.
PREVIOUS Update and expand alert functionality |
NEXT Alert examples |
This documentation applies to the following versions of Splunk® Enterprise: 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15
Comments
So we do have to change the search string on one page and the email action on another?
In response to 0range, the preferred method to edit a search string for an alert is the following:
1. Go the Alerts page.
2. Select Open in Search for the alert you want to modify.
3. Modify the Search.
4. Run the Search.
5. Select Save.
To edit email actions:
1. Select the Alert from Alerts page.
2. For Actions, click Edit.
3. Click Send Email and modify the email actions.