Configure KPI thresholds with machine learning in ITSI

Instead of manually configuring threshold levels or selecting a threshold template that may not fit your historic KPI data, use machine learning-assisted thresholding to receive threshold recommendations tailored to your KPI data. Select the Use recommended thresholding configuration option to receive the optimal time-policies and threshold levels for your data generated by Splunk AI.

The recommended policy will have adaptive thresholding turned on by default, which automatically re-evaluates and updates threshold values as the KPI data changes over time. Recommendations are calculated using the standard deviation algorithm. For more information about adaptive thresholding, see Create adaptive KPI thresholds in ITSI.

Prerequisites

• Install Python for Scientific Computing version 3.0.0 or later in order to use this feature.
• Your KPI needs at least 7 days worth of backfilled data or display a historical pattern or trend in order to produce recommendations. For more information, see When to use adaptive thresholds.

Steps

1. Select the KPI thresholds tab for a KPI. Alternatively, apply AI thresholding to multiple KPIs by running the Run AI auto-tune action on the KPIs tab of the Service and KPI management page.
2. From the Threshold type tab, select the Get recommendations button from the AI thresholding option.
3. Set up the following fields in order to configure the algorithm that generates thresholds for your KPIs:
   • Analysis Window: Set the time period over which your KPI data will be analyzed to detect patterns and behavior. The threshold values and time policies generated by Splunk AI will be based on the data available in this window. For the most accurate recommendations, select a time range that captures the typical behavior of the KPI.

   Note: Selecting 7 days of data will help the algorithm detect daily patterns in KPI data. Selecting 30 days or more (14 days at minimum) helps the algorithm detect weekly patterns, in addition to daily patterns.

   • Thresholding direction: Specify the direction (increase, decrease, or both) that you want threshold level severities to follow, relative to the baseline "normal" KPI value. The algorithm can automatically select the correct thresholding direction based on an analysis of your data.
   • Apply as: Set how you want the thresholds applied. Threshold values can automatically update based on data patterns detected by AI, or remain static.
   • Positive thresholds only: Toggle on so that only positive threshold values are generated.
   • Threshold sensitivity: Set the algorithm's sensitivity to changes in your data before an alert is generated. For example, a higher sensitivity generates threshold values that are closer together and more sensitive to data fluctuations, potentially generating more alerts.
4. After configuring the parameters, select Load recommendations.
5. You will receive a summary of recommended threshold levels, threshold type, and KPI time policies for your KPI. View the settings on the Threshold levels tab.

Next steps

• After you configure KPI thresholds, you can set up alerts to notify you when aggregate KPI severities change. ITSI generates notable events in Episode Review based on the alerting rules you configure. For information, see Receive alerts when KPI severity changes in ITSI.
• Alternatively, you can set up drift detection for the KPI. Drift detection uses machine learning algorithms to automatically detect abnormalities in KPI behavior and notify you in Episode Review. For more information, see Monitor KPI data drift in ITSI.