AlertSite by SMARTBEAR integration 🔗
AlertSite by SMARTBEAR provides advanced synthetic monitoring platform for APIs, mobile and web applications. The Splunk OnCall integration with AlertSite allows you to send alerts into the Splunk OnCall timeline by using the generic email endpoint. The following guide will walk you through this integration.
In Splunk OnCall 🔗
To enable the AlertSite integration in Splunk OnCall to be able to trigger and resolve incidents via email:
In Splunk OnCall, navigate to Settings, then Alert Behavior. Select Integrations and select AlertSite.
If it is not already enabled, select Enable Integration. This will generate an email address to which you can send email alerts. Note down this address. You will need to specify it in the AlertSite application.
The $routing_key should be replaced with the key of a team to which you want to route the alerts. For example:
db212e48-……8669:strong:+databaseteam@alert.victorops.com
Team routing keys are configured at the bottom of the Settings > Integrations page. For details, see Routing Keys <http://help.victorops.com/knowledge-base/routing-keys/>.
If you do not use routing, remove the +$routing_key part, including the plus sign, so that the email looks like this:
db212e48-……8669@alert.victorops.com
Configuring AlertSite 🔗
Customizing Email Templates 🔗
When using the Splunk OnCall email endpoint, the email subject must include specific keywords - CRITICAL or PROBLEM to open a new incident, and RESOLVED or OK to resolve an incident. You can customize AlertSite email alerts to include these keywords.
Note
The email subject line must be the same in both the error and clear templates (apart from the CRITICAL/PROBLEM and RESOLVED/OK words). This is needed for Splunk OnCall to recognize that the “clear” email is related to the incident opened by the “error” email. For example, do not use the $STATUS variable in the subject line, because the status code will be different in the “error” and “clear” alerts, and, in this case, Splunk OnCall will not be able to match these alerts.
To create custom alert templates for Splunk OnCall in AlertSite:
In the AlertSite user interface, navigate to Alerts then Template Editor.
- To configure the error template:
Filter the template list to show only Alert Type: Site Error.
On the list, select the AlertSite Template for Site Error` with the delivery method Text.
Select the template text in the editor to activate the edit mode.
Enter the following: - Template name: Splunk OnCall - Monitor Error (or similar) - Subject: [AlertSite] Monitor Alert - $DESCRIP CRITICAL
Note
You can use another subject, but make sure it includes the word CRITICAL and does not include the $STATUS variable.
Select Save As to save the changes as a new template.
- To configure the ”all clear“ template:
Filter the template list to show only Alert Type: Site Clear.
Select the AlertSite Template for Site Clear with the delivery method of Text.
Select the template text in the editor to activate the edit mode.
Enter the following:
Template name: Splunk OnCall - Monitor Clear (or similar).
Subject: [AlertSite] Monitor Alert - $DESCRIP OK
Note
The subject must be exactly the same as in the error template, but with the OK word instead of CRITICAL.
Select Save As to save the changes as a new template.
You can see the created templates on the template list:
Adding Splunk OnCall to Alert Recipients 🔗
Next, you need to add the Splunk OnCall email endpoint that you generated in In Splunk OnCall as an alert recipient in AlertSite:
In the AlertSite, navigate to Alerts then Alert Recipient.
Select + New Recipient.
- In the dialog that appears, enter the following:
Name: any name, for example, Splunk OnCall
Recipient: the email address you generated in Splunk OnCall
Mode: E-mail (text format)
Click Submit.
Select the created recipient and click Edit Recipient.
On the Availability Alerts tab, set the option to Alert whenever an error clears. This is needed to automatically resolve incidents in Splunk OnCall when an error clears in AlertSite.
Configure other options as needed. For a description of available options, see Recipient Properties - Availability Alerts.
Click Submit.
Assigning Custom Templates to Splunk OnCall Email Alerts 🔗
Now, you need to assign your custom alert templates to the Splunk OnCall alert recipient. To do this, you need to create a recipient group <http://doc.alertsite.com/synthetic/alerts/recipient-groups.htm> that contains the Splunk OnCall recipient, the custom templates, and the monitor whose alerts you want to send to Splunk OnCall:
In AlertSite, navigate to Alerts, then Recipient Groups.
Select New Recipient Group.
Enter a name for the group.
Under Monitors, add the monitors that should send alerts to Splunk OnCall.
Under Recipients, add the Splunk OnCall recipient.
Under Custom Templates, add the Splunk OnCall “error” and “clear” templates that you created earlier.
Optionally: Under Error Types, select specific error codes that should be reported to Splunk OnCall The default value is all errors.
Select Save as new.
Now, the specified monitors will send alerts to Splunk OnCall.
Viewing Alerts in Splunk OnCall 🔗
You can view AlertSite alerts in the Splunk OnCall Timeline and Incidents tabs.
Select More info to view the alert contents, including the description of the error occurred:
The team members can then acknowledge the incidents and take action to resolve them. The incidents will also be resolved automatically when a “clear” notification arrives from AlertSite.