Splunk® IT Service Intelligence

Service Insights Manual

Synchronize KPI searches in ITSI

By default, ITSI staggers the search scheduling of KPIs in order to reduce search load. For example, if you have five KPIs that are scheduled to run every 5 minutes, the search to update the value of each KPI from the summary index is staggered over the 5 minute interval (the first KPI at minute 1, the second KPI at minute 2, and so on).

You can synchronize KPI searches so they update at the same time during the scheduled interval. For example, if 5 KPIs are scheduled to run every 15 minutes, they ALL run at the 15/30/45/00 mark instead of being staggered over the interval.

Prerequisites

  • Only users with file system access, such as system administrators, can synchronize KPI searches.
  • Review the steps in How to edit a configuration file in the Admin Manual.
  • If you are using Splunk Cloud Platform, you must file a support ticket in order to complete the steps below.

Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location.

Steps

  1. Open or create a local itsi_settings.conf in $SPLUNK_HOME/etc/apps/SA-ITOA/local/
  2. Add the following stanza:
    [synced_kpi_scheduling]
    disabled = 0
    

When disabled = 0, newly created KPI saved searches run at the same time during each scheduled interval. After a KPI is updated, its search schedule will be overwritten to follow the synchronized schedule.

To reset the search schedules of all existing KPIs to use the new synchronized search schedule after you set disabled = 0, restart Splunk software and then use mode 4 of the kvstore_to_json.py script. See Regenerate KPI search schedules (mode 4) for details.

This setting affects all KPIs, including base searches, for all services. Enabling synced scheduling can have a significant performance impact because it increases the scheduler load. The increased load can result in a delay in search execution due to the number of searches being dispatched at the same time. You might need to scale up your hardware in order to support the increased load.

Regenerate KPI search schedules (mode 4)

The kvstore_to_json.py mode 4 option regenerates the search schedules for your KPIs. Use this command if you have set your KPI saved search schedules to run at the same time in itsi_settings.conf. Run this command to reset the search schedules of all your KPIs to use the new search schedule. See Synchronize KPI searches in ITSI for more information.

  1. Run kvstore_to_json.py in mode 4.
    For example:
    cd $SPLUNK_HOME
    bin/splunk cmd python3 etc/apps/SA-ITOA/bin/kvstore_to_json.py -m 4
    
  2. Enter the requested information at the prompts (default interactive mode only).
  3. You'll see the following message after the KPI search schedules have been reset:
    Retrieving KPIs to reset their saved search scheduling
    Saving updated KPI scheduling
    Done.
    
Last modified on 21 June, 2024
Create KPI base searches in ITSI   Overview of advanced thresholding in ITSI

This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.11.0, 4.11.1, 4.11.2, 4.11.3, 4.11.4, 4.11.5, 4.11.6, 4.12.0 Cloud only, 4.12.1 Cloud only, 4.12.2 Cloud only, 4.13.0, 4.13.1, 4.13.2, 4.13.3, 4.14.0 Cloud only, 4.14.1 Cloud only, 4.14.2 Cloud only, 4.15.0, 4.15.1, 4.15.2, 4.15.3, 4.16.0 Cloud only, 4.17.0, 4.17.1, 4.18.0, 4.18.1, 4.19.0, 4.19.1


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters