Set alerts on metrics in Splunk IAI

As a Splunk Industrial Asset Intelligence (IAI) administrator, you can set alerts on metrics based on thresholds.

Set alerts to accomplish the following goals:

• Define the numeric ranges and thresholds that indicate whether a metric value is normal (green), warning (yellow), or critical (red) on monitor views.
• Automatically send an email when a metric meets a trigger condition.
• Automatically send an SMS message when a metric meets a trigger condition.

Unlike alerts set on searches in Splunk Enterprise, alerts you set in IAI do not show up in the Searches, Reports, and Alerts list.

Set an alert on a metric

Follow the procedure to set an alert on a metric in Splunk IAI.

Prerequisites

• You must have at least one asset hierarchy with associated metrics data linked to an operation. See Associate metrics data to your asset structure in Splunk IAI.
• Set up and configure the types of actions that you want to happen for an alert.
  • Email actions require that you configure the mail server in Splunk Enterprise. See Configure email notification settings in the Splunk Enterprise Alerting Manual.
  • SMS actions require that you download the Twilio SMS Alerting add-on and configure it according to the instructions. See Details on Splunkbase. This add-on is developer supported.

Steps

1. From Splunk IAI, click the Browse icon.
2. Navigate to the metric that you want to set an alert on.
   • To create an alert on a metric for an asset, navigate to that asset in your asset hierarchy.
     Alerts that you create from a metric on an individual asset, even if that metric is inherited from a group, apply only to that asset.
   • To create an alert on a metric for all assets in a group, click the Groups tab, go to the relevant group, and then click the Group Metrics tab.
     Alerts that you create from the Group Metrics tab apply to all assets in the group.
3. Click the name of the metric. The metric details page opens.
4. Before configuring an alert, right-click Analyze and open that page in a new tab. Confirm that this metric is functioning as expected and data is available.

   Configuring an alert on a metric that is misconfigured or missing data can cause the alert to fail.

   Return to the metric details page.
5. Under Alerts, click Add.
6. Give your alert a Name and an optional Description, and then click Next.
7. Configure the condition to trigger the alert using the options next to When result is.
8. Select a Severity for the alert. This value determines the color displayed in the monitor view widget for this metric when the alert trigger condition is met.
   • Normal displays a green widget.
   • Warning displays a yellow widget.
   • Critical displays a red widget.
9. Set an alert suppression duration to indicate how long Splunk IAI must wait before triggering the alert again. This setting takes effect only if you specify an action in the next screen.
10. Click Next
11. (Optional) Click + Add Action to select one or more alert actions to happen when the alert triggers.
    • If you add the Send email action, complete the fields.
    • If you add the Send Twilio SMS Alerts action, complete the fields.
12. Click Finish.

Alert frequency

Alerts in Splunk IAI are managed by a modular input that evaluates threshold conditions every minute. You can adjust that interval by editing the cron schedule in the input_alerts_conditions_manager modular input.

1. From Splunk Web, select Settings > Data Inputs.
2. Select the input_alerts_conditions_manager input.
3. Click thresholdanalyzer.
4. Select the More settings check box.
5. Make changes to the Interval cron schedule.

   Increasing the frequency of the modular input could negatively affect Splunk IAI performance.

6. Click Save.

Alert precedence

You can create multiple alerts on the same metric. You can save as many alerts as you want, including duplicate alerts or alerts with contradictory settings for severity thresholds.

For metrics with multiple alerts and different severity thresholds, the severity that displays on a monitor view widget for the metric reflects alerts set to run earlier. Alerts set to run earlier in a day, for example, take precedence over alerts set to run later in a day.

Edit an alert

As an IAI administrator, you can edit any alert.

1. From Splunk IAI, click the Browse icon.
2. Navigate to the metric that you want to edit an alert for.
   • If the alert you want to edit is set on a metric for an asset, navigate to that asset in your asset hierarchy to display its metrics.
   • If the alert you want to edit is set on a metric for a group, click the Groups tab, select the relevant group, and then click the Group Metrics tab.
3. Click the name of the metric. The metric details page opens.
4. Under Alerts, find the alert you want to edit and click the pencil icon.
5. Edit the alert and save it when finished.

Delete an alert

As an IAI administrator, you can delete any alert.

1. From Splunk IAI, click the Browse icon.
2. Navigate to the metric that you want to delete an alert for.
   • If the alert you want to delete is set on a metric for an asset, navigate to that asset in your asset hierarchy to display its metrics.
   • If the alert you want to delete is set on a metric for a group, click the Groups tab, select the relevant group, and then click the Group Metrics tab.
3. Click the name of the metric. The metric details page opens.
4. Under Alerts, find the alert you want to delete and click the trash icon.
5. In the confirmation box, click Delete.