Configure alert notifications in Splunk App for Infrastructure

Configure an entity or group alert to send a notification when an entity or group meets or exceeds a certain threshold. You can configure these types of alert notifications:

• Email
• VictorOps for Splunk

You can include multiple alert notification methods for each alert, and mix and match alert notifications for alert thresholds. For example, you can create two alert notifications with different notification methods that share the same alert threshold.

Configure email notifications

SAI uses Splunk Enterprise email notification settings to send email notifications when alerts meet or exceed certain thresholds. For more information about configuring email notification settings, see Email alert action in the Splunk Enterprise Alerting Manual.

Configure VictorOps for Splunk notifications

VictorOps For Splunk is automated incident management software that aligns log management, monitoring, and chat tools to automate the delivery of alert notifications. When you integrate VictorOps with SAI, you can create and manage alerts in VictorOps to notify a designated person or on-call team with information about a triggered SAI alert.

Prerequisites

• You have administrator capabilities in VictorOps.
• You configured the Splunk integration in VictorOps. For more information, see the Splunk Integration Guide on the VictorOps website.

Steps

Follow these steps to integrate SAI notifications with VictorOps.

1. In VictorOps, get your API Key and Routing Key. If you need help finding the API Key and Routing Key, see the Splunk Integration Guide on the VictorOps website.
2. In Splunk Web, open SAI and go to Settings > Notifications.
3. Under VictorOps settings, enter a unique Name to identify the integration. You can't edit the name after you create it. If you want to edit the name, you have to remove the configuration and create a new one.
4. Enter your Splunk VictorOps API Key and Routing Key.
5. Click Save Credentials. When you save the credentials, SAI sends a test notification to your VictorOps timeline.
6. Verify the authentication of SAI in Splunk VictorOps. Go to your Splunk VictorOps timeline and confirm you received a notification from SAI. The test notification looks like this:

   Splunk SII, Info: Test verification integration.
   

7. If you didn't receive this notification, check your API Key and Routing Key and retry saving your credentials again.

Create and send a VictorOps alert notification

For information about creating and sending an alert using VictorOps, see Create and modify alerts in Splunk App for Infrastructure.

Remove VictorOps credentials

You can have only one integration with VictorOps in SAI at a time. To remove an integration, delete the saved API Key and Routing Key in SAI.

1. In Splunk Web, open SAI and go to Settings > Notifications.
2. Under VictorOps settings, click Remove Credentials.