Splunk® IT Service Intelligence

Install and Upgrade Manual

Acrobat logo Download manual as PDF

Splunk IT Service Intelligence version 4.5.x will no longer be supported as of April 29, 2022. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see Before you upgrade IT Service Intelligence.
This documentation does not apply to the most recent version of ITSI. Click here for the latest version.
Acrobat logo Download topic as PDF

Before you upgrade IT Service Intelligence

Perform the steps in this topic before you upgrade IT Service Intelligence (ITSI) to the latest release. Splunk Cloud customers must work with Splunk Support to coordinate upgrades to ITSI. Version 4.5.x supports upgrading from version 4.2.x or later. To upgrade from earlier versions, perform intermediary upgrades.

Remove episode lookup entries from transforms.conf

The instructions field was added to the itsi_notable_group_user KV store collection in version 4.5.0. Before upgrading to version 4.5.0 or later from a pre-4.5.0 version, remove any entries for itsi_notable_group_user_lookup in your local transforms.conf so this value can be populated in the user KV store collection.

Copy any changes to itsi_rules_engine.properties

As of version 4.4.0, you can make changes to a local copy of the itsi_rules_engine.properties file at $SPLUNK_HOME/etc/apps/SA-ITOA/local/ and these changes will take precedence over the default file. Previously, this file was not treated like a regular Splunk configuration file, so changes to a local copy of the file had no impact.

If you've made changes to the default file in the past, make a copy of these changes before upgrading to version 4.4.0 or higher from a pre-4.4.0 version. After you upgrade, create a blank itsi_rules_engine.properties file at $SPLUNK_HOME/etc/apps/SA-ITOA/local/ and add these changed settings to the local file. This step ensures that your changes to the file will persist through future upgrades.

Make all future changes to itsi_rules_engine.properties in the local file rather than the default file. For the contents of the file, see Rules Engine properties reference in ITSI in the Event Analytics Manual.

For more information about how Splunk handles changes to configuration files, see Configuration file precedence in the Splunk Enterprise Admin Manual.

Trim down episode KV store collections

Upgrades to version 4.4.2 and higher might be slower than usual for the following reasons:

  • The mod_time field is added to all existing objects in the itsi_notable_group_user KV store collection.
  • Episode comments are migrated from the itsi_notable_event_comment collection to the itsi_grouped_alerts index.

To prevent slow upgrades, trim down the Event Analytics KV store collections to less than 500,000 objects before upgrading to version 4.4.2. After trimming these collections, the deleted episodes will no longer appear in Episode Review.

Performing these steps on versions prior to 4.4.x will delete the objects from the KV store. These deleted entries are not archived.


You must have the itoa_admin role to delete objects from these KV store collections. For more information, see KV store collection permissions in ITSI.


  1. Check the number of objects in the itsi_notable_group_system KV store collection. If there are more than 500,000 objects, trim the collection to less than 500,000.
    1. Open or create a local copy of itsi_notable_event_retention.conf at $SPLUNK_HOME/etc/apps/SA-ITOA/local/.
    2. Add the following stanza:
      # 30 days
      retentionTimeInSec = 2592000
    3. Reduce the retentionTimeInSec setting based on approximately how long it takes for your system to generate 500,000 episodes. For example, if it takes 15 days to generate 500,000 episodes, the retention time in seconds would be 1296000.
  2. Set the data type of the mod_time field as time for the following KV store collections.
    1. Open or create a local copy of collections.conf at $SPLUNK_HOME/etc/apps/SA-ITOA/local/.
    2. Add the following stanzas:
      field.mod_time = time
      accelerated_fields.mod_time = {"mod_time": 1}
      field.mod_time = time
      accelerated_fields.mod_time = {"mod_time": 1} 
  3. Either wait an hour for the modular input to run, or restart your Splunk software to run it immediately.
  4. Include additional fields for the itsi_notable_group_user command to support.
    1. Open or create a local copy of transforms.conf at $SPLUNK_HOME/etc/apps/SA-ITOA/local/.
    2. Add the following stanza:
      fields_list = _key, status, severity, owner, event_identifier_hash, object_type, mod_time
  5. After trimming the itsi_notable_group_system collection, run the following SPL search to remove objects from itsi_notable_group_user that don't exist in itsi_notable_group_system:

    | inputlookup itsi_notable_group_system_lookup | fields _key | rename _key as id | lookup itsi_notable_group_user_lookup _key as id OUTPUT owner severity status event_identifier_hash object_type mod_time | rename id as _key | outputlookup itsi_notable_group_user_lookup

  6. Check the number of comments in the itsi_notable_event_comment collection. If it's more than 1 million, trim down the collection. The following search trims the comments to the last 90 days:

    | inputlookup itsi_notable_event_comment_lookup | where mod_time > now() - 3*30*24*3600 | eval object_type="notable_event_comment" | outputlookup itsi_notable_event_comment_lookup

  7. After you trim all the collections, revert the changes to collections.conf and transforms.conf.

Make sure no service templates are syncing

If any service templates are syncing when you upgrade ITSI, the upgrade fails. Check the sync status of service templates by clicking Configuration > Service Templates from the ITSI main menu.

Back up the search head

Take a full backup of the search head. For instructions, see Overview of backing up and restoring ITSI KV store data. To back out of the upgrade, you must restore the prior version of Splunk IT Service Intelligence from a backup.

Check admin role inheritance

Make sure the Splunk admin role inherits from the itoa_admin role. The default settings for admin role inheritance for ITSI are contained in authorize.conf. Problems can occur when these settings have been modified in a local version of the file.

Check KV store size limits

The limit of a single batch save to a KV store collection is 500 MB. Check the total amount of data that your services contain, and, if necessary, increase the KV store size limit in $SPLUNK_HOME/etc/apps/SA-ITOA/local/limits.conf. This setting controls the maximum size, in megabytes (MB), of the results that are returned for a single query to a collection.


  • Only users with file system access, such as system administrators, can increase the KV store size limit.
  • Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.

Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location.


  1. Open or create a local limits.conf file in $SPLUNK_HOME/etc/apps/SA-ITOA/local/.
  2. Increase the max_size_per_batch_save_mb value in the [kvstore] stanza:
    max_size_per_batch_save_mb = [new value]

Review known issues and changes

Review the following topics before you upgrade ITSI:

  1. Compatible versions of the Splunk platform. See Splunk Enterprise system requirements.
  2. Hardware requirements. See Planning your hardware requirements.
  3. Known issues with the latest release of IT Service Intelligence. See Known issues in Splunk IT Service Intelligence in the Release Notes.
  4. Removed features in the latest release of IT Service Intelligence. See Removed features in the Release Notes.

Recommendations for upgrading IT Service Intelligence

Upgrade both the Splunk platform and IT Service Intelligence in the same maintenance window. See the Splunk Enterprise system requirements to verify which versions of Splunk ITSI and Splunk Enterprise are supported with each other.

If you're upgrading to a Python 3 release of Splunk Enterprise (version 8.x), you must upgrade ITSI and all other apps before upgrading Splunk Enterprise. For more information, see Python 3 migration with ITSI.

  1. Upgrade Splunk Enterprise to a compatible version.
  2. Upgrade Splunk platform instances.
  3. Upgrade Splunk IT Service Intelligence.
  4. Review, upgrade, and deploy add-ons.
  5. See Version-specific upgrade notes for post-installation tasks.

Upgrading ITSI deployed on a search head cluster is a multi-step process. The procedure is detailed in Upgrade IT Service Intelligence in a search head cluster environment in this manual.

Last modified on 08 November, 2020
Uninstall Splunk IT Service Intelligence
Steps to address the Apache Log4j vulnerabilities in ITSI or IT Essentials Work

This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.5.0 Cloud only, 4.5.1 Cloud only

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters