Synchronize KPI searches in ITSI
By default, ITSI staggers the search scheduling of KPIs in order to reduce search load. For example, if you have five KPIs that are scheduled to run every 5 minutes, the search to update the value of each KPI from the summary index is staggered over the 5 minute interval (the first KPI at minute 1, the second KPI at minute 2, and so on).
You can synchronize KPI searches so they update at the same time during the scheduled interval. For example, if 5 KPIs are scheduled to run every 15 minutes, they ALL run at the 15/30/45/00 mark instead of being staggered over the interval.
- Only users with file system access, such as system administrators, can synchronize KPI searches.
- Review the steps in How to edit a configuration file in the Admin Manual.
- If you are using Splunk Cloud Platform, you must file a support ticket in order to complete the steps below.
Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location.
- Open or create a local itsi_settings.conf in
- Add the following stanza:
[synced_kpi_scheduling] disabled = 0
disabled = 0, newly created KPI saved searches run at the same time during each scheduled interval. After a KPI is updated, its search schedule will be overwritten to follow the synchronized schedule.
To reset the search schedules of all existing KPIs to use the new synchronized search schedule after you set
disabled = 0, restart Splunk software and then use mode 4 of the
kvstore_to_json.py script. See Regenerate KPI search schedules (mode 4) for details.
This setting affects all KPIs, including base searches, for all services. Enabling synced scheduling can have a significant performance impact because it increases the scheduler load. The increased load can result in a delay in search execution due to the number of searches being dispatched at the same time. You might need to scale up your hardware in order to support the increased load.
Regenerate KPI search schedules (mode 4)
kvstore_to_json.py mode 4 option regenerates the search schedules for your KPIs. Use this command if you have set your KPI saved search schedules to run at the same time in itsi_settings.conf. Run this command to reset the search schedules of all your KPIs to use the new search schedule. See Synchronize KPI searches in ITSI for more information.
kvstore_to_json.pyin mode 4.
cd $SPLUNK_HOME bin/splunk cmd python3 etc/apps/SA-ITOA/bin/kvstore_to_json.py -m 4
- Enter the requested information at the prompts (default interactive mode only).
- You'll see the following message after the KPI search schedules have been reset:
Retrieving KPIs to reset their saved search scheduling Saving updated KPI scheduling Done.
Create KPI base searches in ITSI
Overview of advanced thresholding in ITSI
This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.9.0, 4.9.1, 4.9.2, 4.9.3, 4.9.4, 4.9.5, 4.9.6, 4.10.0 Cloud only, 4.10.1 Cloud only, 4.10.2 Cloud only, 4.10.3 Cloud only, 4.10.4 Cloud only, 4.11.0, 4.11.1, 4.11.2, 4.11.3, 4.11.4, 4.11.5, 4.11.6, 4.12.0 Cloud only, 4.12.1 Cloud only, 4.12.2 Cloud only, 4.13.0, 4.13.1, 4.13.2, 4.14.0 Cloud only, 4.14.1 Cloud only, 4.14.2 Cloud only, 4.15.0, 4.15.1, 4.15.2, 4.16.0 Cloud only
Feedback submitted, thanks!