Receive alerts when KPI severity changes in ITSI

Enable alerting on a single key performance indicator in IT Service Intelligence (ITSI) so you can be alerted when aggregate KPI threshold values change. ITSI generates notable events in Episode Review based on the alerting rules you configure. Use these alerts to investigate and take action on the severity changes of your individual KPIs before they negatively impact the service as a whole.

When you enable alerting on the KPIs in a service template, you must explicitly choose the All KPIs option when you save the template in order for the changes to propagate to the KPIs in the linked services. For more information, see Update a service template in ITSI.

Prerequisites

• You must have write access to the service in order to enable KPI alerting.
• You must create a KPI within a service and configure thresholds for it before you can enable alerting. For more information, see Step 7: Set Thresholds in the KPI configuration workflow.

Steps

1. Click Configuration > Service and open the service the KPI belongs to.
2. On the KPIs tab, select the KPI you want to receive alerts about.
3. Expand the Thresholding panel.
4. Toggle the switch next to Enable KPI Alerting in the Aggregate Thresholds section.
5. Configure the specific severity changes you want to monitor:
   • To receive an alert every single time the KPIs severity changes, select Trigger a notable event for ALL KPI severity changes.
   • To only receive alerts when specific severity changes occur (for example, a change from High to Critical), select Trigger a notable event for specific changes and configure the alerting rules.
6. When you're satisfied with your alerting rules, click Save.

Example alert configuration

You want your analysts to be alerted when a KPI in the Middleware service degrades so they can take the necessary steps to fix it before it affects the service as a whole. You want them to be notified of each severity change over the course of the degradation so they know if things are getting worse.

You create the following alerts:

Trigger a notable event for specific changes

KPI severity changes to Critical from High
KPI severity changes to High from Medium
KPI severity changes to Medium from Low

After an analyst fixes the episode, you want them to receive a final notification that the KPI severity is back to a normal level.

You create the following alert:

KPI severity changes to Normal from Critical, High, Medium, Low

Configure actions for KPI alerts

IT Service Intelligence uses the KPI Alerting Policy to group individual KPI alerts into episodes in Episode Review. By default, this notable event aggregation policy does not contain any action rules. Add action rules to take specific actions on each episode.

1. From the ITSI main menu, click Configuration > Notable Event Aggregation Policies.
2. Open the KPI Alerting Policy.
3. Click the Action Rules tab.
4. Click Add Rules and add one or more action rules for KPI alerts.
5. Click Save to save the policy.

For example, you might configure an action rule to make sure an episode's severity changes to Critical when lots of KPI alerts are coming in.

Disable grouping of individual KPI alerts

By default, ITSI uses the KPI Alerting Policy to group individual notable events received from KPI alerts into episodes in Episode Review. Events are grouped according to the service they belong to, and ITSI breaks an episode if no events are received for one hour. The severity of each episode is determined by the severity of the first event in the episode.

To view the individual notable events being generated, click the gear icon in Episode Review and disable Episode View.

To turn off this grouping behavior altogether and only display individual notable events for KPI alerts, perform the following steps:

1. From the ITSI main menu, click Configuration > Notable Event Aggregation Policies.
2. Find the KPI Alerting Policy in the list of available policies and disable it in the Status column.

The Normalized Policy (Splunk App for Infrastructure) will also create episodes containing individual KPI alerts because it looks for similar event fields when grouping. The Normalized Policy is disabled by default unless you enable it to integrate with the Splunk App for Infrastructure.