Set up alerts in savedsearches.conf

Set up alerts in savedsearches.conf

Configure alerts with savedsearches.conf. Use the $SPLUNK_HOME/etc/system/README/savedsearches.conf.example as an example, or create your own savedsearches.conf. Edit this file in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work.

Follow these steps:

1. Create a saved search.

2. Schedule the search.

3. Define alert conditions.

4. Configure alert actions.

You can set up an alert at the time you create a saved search, or add the alert configurations to your saved search stanza later.

Note: You must have email enabled on your Splunk server for alerts to be sent out. Alternately, your Splunk server must be able to contact your email server. Configure email settings in Manager.

Create a saved search

First, set up a saved search. You can also set up a saved search via savedsearches.conf.

Schedule the search

Next, schedule your search. This means your search runs on the specified schedule. For example, Splunk runs your search every hour or at midnight. If your search meets the alert conditions, then Splunk alerts you.

Add the following attribute/value pairs to your saved search stanza to run the search on a schedule:

userid = <integer>

• UserId of the user who created this saved search.
  • Splunk needs this information to log who ran the search, and create editing capabilities in Splunk Web.
• Possible values: Any Splunk user ID.
• User IDs are found in $SPLUNK_HOME/etc/passwd.
  • Look for the first number on each line, right before the username.
  • For example 2:penelope....

enableSched = < 0 | 1 >

• Set this to 1 to enable schedule for search
• Defaults to 0.

schedule = <string>

• Cron style schedule.
• For example, */12 * * * *).

Note Cron lets you use standard cron notation to define your scheduled search interval. In particular, cron can accept this type of notation: 00,20,40 * * * *, which runs the search every hour at hh:00, hh:20, hh:40. Along the same lines, a cron of 03,23,43 * * * * runs the search every hour at hh:03, hh:23, hh:43. Splunk recommends that you schedule your searches so that they're staggered over time. This reduces system load. Running all of them (*/20) every 20 minutes means they would all launch at hh:00 (20, 40) and might slow your system every 20 min.

execDelay = <integer>

• Amount of time (in seconds) from most recent event to the execution of the scheduled search query.
• Defaults to 0.

Alert conditions

Now define alert conditions. Alert conditions tell Splunk whether or not to send you an alert. Enter a threshold number of events, sources, or hosts in your results. If the alert conditions are met, Splunk notifies you via email or triggers a shell script.

counttype = <string>

• Set the type of count for alerting.
• Possible values: number of events, number of hosts, number of sources, number of sourcetypes.

relation = <string>

• How to compare against counttype.
• Possible values: greater than, less than, equal to, drops by, rises by.

quantity = <integer>

• Number to compare against the given counttype.

So if you have the following:

counttype = number of events
relation = rises by
quantity = 25

Splunk alerts you if your search results have risen by 25 since the last time the search ran.

You can also define a search condition for your alerts:

alert_condition = <string>

• A search that is evaluated on the artifacts of the saved search to determine whether to trigger the alerts.
• Alerts are triggered if the search specified yields a non-empty search result list.

Configure email alert actions

Tell Splunk to send an email once an alert is triggered.

action.email = 0 | 1

• whether the action is enabled or disabled

Configure the format of the email that is sent. These will override the settings that are defined in alert_actions.conf:

action.email.to = <string>

• Comma separated list of email addresses to send alerts to.

action.email.from = <email address>

• email address that would be used as the sender's address

action.email.subject = <string>

• the subject of the email delivered to recipients

action.email.mailserver = <string>

• address of the MTA server to be used to send emails

Example

This example runs a search for events containing the term "sudo" on a schedule, and sends the results via email.

[sudoalert]
action.email = 1
action.email.to = me@work.org 
action.email.from = splunk@work.org
action.email.subject = Sudo Alert!
action.email.mailserver = mail.work.org
counttype = number of events
enableSched = 1
quantity = 10
search = sudo
relation = greater than
schedule = */12 * * * *
role = Admin

• Was this topic useful?
• Post a comment