Splunk® IT Service Intelligence

Release Notes

This documentation does not apply to the most recent version of Splunk® IT Service Intelligence. For documentation on the most recent version, go to the latest release.

Known issues in Splunk IT Service Intelligence

This version of IT Service Intelligence (ITSI) has the following known issues and workarounds.

Adaptive Thresholding

Date filed Issue number Description
2024-09-04 ITSI-37270 Use Recommended Thresholding Configuration cannot use all backfilled events

Workaround:
While backfilling the KPI customer can set the fill data gaps option other than Template:Last available value and after backfill completes successfully they can switch the option to Template:Last available value.

Backup/Restore and Migration Issues

Date filed Issue number Description
2023-10-12 ITSI-32459 Cleanup the migration_helper folder before the restore of the backup starts
2023-04-19 ITSI-29586 Unable to restore default scheduled backup

Workaround:
Download the Default Scheduled Backup and restore the downloaded backup

Deep Dive

Date filed Issue number Description
2023-12-19 ITSI-33470 Filter for Deep dives and Glass tables lister not working as expected with equal sign

Entities

Date filed Issue number Description
2023-08-23 ITSI-31855, ITSI-33386 API entity_discovery_searches Failed to return discovery searches post upgrade

Workaround:
Once all the discovery searches related to the entity ran once, this issue will not exist.

If the problematic search is 'disabled' and not intended to run anymore, can utilize the clean up command to clean this search out. ([1] )

If the problematic search simply has a run time that is much further in the future, then, you can change the cron schedule and let it run sooner and then change the time back. this way, you force the search to run again so the new status format gets saved.

2023-08-14 ITSI-31723 Error modal appears when user attempts to filter entities with a parenthesis in the name on entity management page

Workaround:
Use backslash before the special character. To search for "myhost(" try "myhost\("
2023-04-19 ITSI-29586 Unable to restore default scheduled backup

Workaround:
Download the Default Scheduled Backup and restore the downloaded backup

Entity Rules

Date filed Issue number Description
2024-05-06 ITSI-35571 New entities are not added to linked services even if they match the filter conditions

Notable Events

Date filed Issue number Description
2024-08-30 ITSI-37188 ITSI does not check to see if ServiceNow Incident exists before attempting to close it
2024-06-20 ITSI-36397 Actions are not performed if the event is breaking the episode based on the timebased criteria and grouping into a new group.

Workaround:
Reset Template:Policy rules check frequency delay to 60000 under Template:$SPLUNK HOME/etc/apps/SA-ITOA/local/itsi rules engine.properties

OR

Update the action rules in the new to perform the action when Template:Number of events in this episode is less than or equals to 2 if the action is Template:Number of events in this episode is == 1

2024-02-13 ITSI-34430 Groups Restore in rules engine should not be done based on the bidirectional ticketing events
2024-02-08 ITSI-34393 BDT event should not satisfy 'if episode is broken' action rule for inactive episodes
2023-11-26 ITSI-33166 Rules Engine process gets enabled after Splunk restart even if it is disabled

Workaround:
Enable High Scale EA Modular input under{{ Setting -> Data Inputs -> IT Service Intelligence High Scale Event Analytics Modular Input}}
2023-10-27 ITSI-32723 Newlines are converted to spaces when posting a comment to an Episode
2023-08-04 ITSI-31559 The Preview Mode is not showing results when smart mode is turned on

Workaround:
We can replace the below Template:Itsicorrelationsearch stanza in Template:Apps/SA-ITOA/package/local/commands.conf

{noformat}[itsicorrelationengine] type = custom command.arg.1=-J-Xmx8192M command.arg.2=-Dlog4j.configurationFile=../default/log4j_correlation_engine.xml command.arg.3=-J-XX:+UseG1GC command.arg.4=-DitsiCorrelationEngine.configurationFile=../default/itsi_correlation_engine.properties command.arg.5=-Dfile.encoding=UTF-8 run_in_preview = false chunked = true{noformat}

and disable and enable the Template:Itsi event grouping search job ( restart the rules engine process)

2023-06-19 ITSI-31057 host field value not visible to Rules Engine
2023-05-12 ITSI-30099 When multiple actions are triggered, field does not get updated according to the last action rule

Notable Event Aggregation Policies

Date filed Issue number Description
2024-08-30 ITSI-37188 ITSI does not check to see if ServiceNow Incident exists before attempting to close it
2024-06-20 ITSI-36397 Actions are not performed if the event is breaking the episode based on the timebased criteria and grouping into a new group.

Workaround:
Reset Template:Policy rules check frequency delay to 60000 under Template:$SPLUNK HOME/etc/apps/SA-ITOA/local/itsi rules engine.properties

OR

Update the action rules in the new to perform the action when Template:Number of events in this episode is less than or equals to 2 if the action is Template:Number of events in this episode is == 1

2024-02-13 ITSI-34430 Groups Restore in rules engine should not be done based on the bidirectional ticketing events
2024-02-08 ITSI-34393 BDT event should not satisfy 'if episode is broken' action rule for inactive episodes
2023-11-26 ITSI-33166 Rules Engine process gets enabled after Splunk restart even if it is disabled

Workaround:
Enable High Scale EA Modular input under{{ Setting -> Data Inputs -> IT Service Intelligence High Scale Event Analytics Modular Input}}
2023-10-27 ITSI-32723 Newlines are converted to spaces when posting a comment to an Episode
2023-08-04 ITSI-31559 The Preview Mode is not showing results when smart mode is turned on

Workaround:
We can replace the below Template:Itsicorrelationsearch stanza in Template:Apps/SA-ITOA/package/local/commands.conf

{noformat}[itsicorrelationengine] type = custom command.arg.1=-J-Xmx8192M command.arg.2=-Dlog4j.configurationFile=../default/log4j_correlation_engine.xml command.arg.3=-J-XX:+UseG1GC command.arg.4=-DitsiCorrelationEngine.configurationFile=../default/itsi_correlation_engine.properties command.arg.5=-Dfile.encoding=UTF-8 run_in_preview = false chunked = true{noformat}

and disable and enable the Template:Itsi event grouping search job ( restart the rules engine process)

2023-06-19 ITSI-31057 host field value not visible to Rules Engine
2023-05-12 ITSI-30099 When multiple actions are triggered, field does not get updated according to the last action rule

Glass Table

Date filed Issue number Description
2024-02-27 ITSI-34543 GlassTables charts not showing null values in results

Workaround:
the workaround is to exclude null from search results (but it's not a workaround as the customer wants to include what's the return of the null values )
2023-12-19 ITSI-33470 Filter for Deep dives and Glass tables lister not working as expected with equal sign
2023-01-10 ITSI-27969 Ad hoc search should work properly even if we add it after deleting the existing the kpi data source from the visualization

Workaround:
Remove the value of options field from glass table source code in visualization when you delete the KPI data source and add adhoc data source in same visualization.

KPI Base Searches

Date filed Issue number Description
2023-12-11 ITSI-33323 Using a Service with Shared Base Searches KPIs. But extra entities are being added to the KPI results.

Workaround:
Re-adding any of the entity filter would sync the mapping of Services and Entities
2023-02-23 ITSI-28869 Adhoc searches should not be validated while creating KPI base with metrics search option

KPI Search Calculation

Date filed Issue number Description
2023-09-13 ITSI-32031 itsi_at_search_kpi_minus7d Missing field alert_value at time
2023-06-06 ITSI-30550 Show search/parser errors in UI for KPI creation model

Maintenance Window

Date filed Issue number Description
2023-10-18 ITSI-32628 Maintenance Window duration drop down is not updating End time value upon changing duration
2023-08-17 ITSI-31755 Incorrect default start time for the maintenance window

Performance

Date filed Issue number Description
2023-08-02 ITSI-31548, ITSI-36787 App SA-ITSI-AT-Recommendations failing Python3 readiness check

Workaround:
The customer is seeing a failing python upgrade readiness scan because we are coding exclusively in Python 3. We are not supporting Python 2 and as such, we were not previously using libraries that support Python 2 and Python 3 simultaneously (like Template:Six and Template:Future). Since Python for Scientific Computing is a requirement for Assisted Thresholding, we are already guaranteeing that the user will have Python 3 installed (our app gets Python from PSC, not from Splunk).

For future reference, code from the Template:SA-ITSI-AT-Recommendations app does *not* need to pass the Python upgrade readiness scan

Role Based Access Controls

Date filed Issue number Description
2023-05-04 ITSI-30017 A user in itoa_user role cannot open ITSI homeview in SHC.

Workaround:
We have to add the list_search_head_clustering capability to the default authorize.conf.

Service Analyzer

Date filed Issue number Description
2023-10-02 ITSI-32214 Service analyzer link for service does not show up
2023-09-18 ITSI-32093, ITSI-31750 In ITSI 4.17, when a user has no quota left to run a concurrent search to populate, the search now fails instead of queuing. This leads to errors in the Service Analyzer.

Workaround:
Increase search quotas (and dispatch size) for the role. Update the concurrent search quota by updating the settings srchJobsQuota and possibly rtSrchJobsQuota in authorize.conf for the appropriate roles.
2023-06-09 ITSI-30822 ITSI degraded-entities-search-manager may have caused indexers cluster to crash

Workaround:
If Service Analyser is running for more than 1 week's time range and search is going through millions of events try to limit the service analyser time range to less than 1 week to limit the search time range.
2023-06-07 ITSI-30580 When the dbconnect app is installed, non-admin ITSI users cannot access their homepage but are routed to the upgrade page.

Workaround:
Add the db_connect_read_app_conf capability to the custom user with a non-admin role. Enable this capability in the default authorize.conf file.

Service Definition

Date filed Issue number Description
2024-01-23 ITSI-34074 Unable to create a service when opening the create service modal before the service template API call is done

Service Health Score

Date filed Issue number Description
2024-01-15 ITSI-33760 Service health score is not getting calculated properly for 'itsi_summary_metrics' index after changing the importance of KPIs

Workaround:
# Replace the Template:Reorganize metrics healthscore results macro's definition with

{noformat}rename itsi_kpi_id AS kpiid, itsi_service_id AS serviceid | fields kpiid, serviceid, urgency, alert_level, alert_name, service, is_service_in_maintenance, kpi{noformat}

  1. Replace the Template:Service health metrics monitor savedsearch's search with

{noformat}| mstats latest(alert_level) AS alert_level WHERE `get_itsi_summary_metrics_index` AND `service_level_max_severity_metric_only` by itsi_kpi_id, itsi_service_id | lookup kpi_alert_info_lookup alert_level OUTPUT severity_label AS alert_name | `mark_services_in_maintenance` | `join_kpi_info(itsi_kpi_id)` | `reorganize_metrics_healthscore_results` | gethealth | `get_info_time_without_sid` | lookup service_kpi_lookup _key AS itsi_service_id OUTPUT sec_grp AS itsi_team_id | fields - alert_severity, color, kpi, kpiid, serviceid, severity_label, severity_value | rename health_score AS service_health_score | eval is_null_alert_value=if(service_health_score="N/A", 1, 0), service_health_score=if(service_health_score="N/A", 0, service_health_score){noformat}

Uncategorized issues

Date filed Issue number Description
2024-04-02 ITSI-34892 command="suppressalert", argument 1 must be an iterator
2024-02-29 ITSI-34551 Breaking event does not trigger breaking action rules
2024-01-23 ITSI-34041 ITSI Episode view triggers a search to populate linked tickets, that is looking back to epoch time=1 second till now
2023-12-25 ITSI-33583 Episode review timeline should be updated when the policy filter is applied
2023-12-18 ITSI-33443 The error message "Entities are unstable in IT Essential works app of Preprod (pp-ritsl)" appears even after updating to the latest version.

Workaround:
Admins should run the following cleanup command:


| cleanupentitydiscoverysearches search_ids="ITSI Import Objects - itsi_entity_name_normalizer".


sc_admins should complete the following:

  1. Go to the Searches Reports, and Alerts page and search for the saved search titled, entity_discovery_search_cleaner.
  2. Edit the cron schedule to run every minute temporarily - Template:* * * * * and enable the search.
  3. After the entities seem stable, set back the schedule to run once everyday.

2023-12-08 ITSI-33286 The status value get reset to In progress if the episode in acknowledged
2023-11-19 ITSI-33134 When the episode from 2nd page is selected and the table refreshed The focus from the episode is getting lost

Workaround:
The workaround for the issue mentioned here is, adding/updating the Template:Itsi notable group lookup macro from the Template:Etc/apps/SA-ITOA/local/macros.conf file.

Add {{itsi_policy_id | eval policy_id=itsi_policy_id, _itsi_is_group_broken=if(is_active==0,1,0)}} at the end of the definition of the Template:Itsi notable group lookup macro so that it would look like below.

{noformat}[itsi_notable_group_lookup] args = definition = lookup itsi_notable_group_user_lookup _key AS itsi_group_id OUTPUT owner severity status instruction | lookup itsi_notable_group_system_lookup _key AS itsi_group_id OUTPUT title description start_time last_time is_active event_count itsi_policy_id | eval policy_id=itsi_policy_id, _itsi_is_group_broken=if(is_active==0,1,0){noformat}

This will make sure that the episode focus will not be lost for another 2 filters Template:Policy and Template:Show Episodes . So, for the filters Status, Owner, Severity, Policy and Show Episodes the episode focus will not be lost.

2023-11-15 ITSI-33113 Bulk Acknowledge Episode can be executed for already Acknowledged episode by another user
2023-11-09 ITSI-33057 The loadjob search is failing when adding "Event Fields" filter on episode review page
2023-10-19 ITSI-32657 Events not being indexed into itsi_tracked_alerts if SSL in not enabled

Workaround:
Go to Data Inputs -> HTTP Event Collector -> "Enable SSL" checkbox → Enable It
2023-10-17 ITSI-32621 Notable drildown search not working on Episode view with sourcetype and colon
2023-10-13 ITSI-32474, ITSI-34680 Adding Ticketing filter after editing columns in Episode Review and saving does not respect the saved state

Workaround:
Reload the Episode Reviev page after the save to reload the Event management state
2023-10-09 ITSI-32413, ITSI-32000 Wrong activity message while running action, when configured Hybrid Action Dispatch
2023-10-09 ITSI-32409 Policy filter does not showed up in the Episode Review eventhough user has access to policy details view
2023-10-05 ITSI-32375, ITSI-30097 Alert Breakdown chart on entity type health page should shows other count
2023-10-01 ITSI-32208 The "itoa_interface/service" endpoint is not returning any results
2023-09-21 ITSI-32156 preview results not working while NEAP creation in windows setup
2023-09-05 ITSI-31978 Correlation search edit page malfunctions when time range set to "All Time"

Workaround:
*Workaround 1*
  1. From Setting-->Searches, Reports, and Alerts-->search for CS-->Edit Alert-->Update time range to Template:Last 15 Minutes from Template:All Time
  • Workaround 2*
  1. Update/add itsi/local/savedsearches.conf as following:

{noformat}dispatch.earliest_time = 0 dispatch.latest_time = now{noformat}

2023-09-04 ITSI-31923 After Changing Splunkd Custom Management Port, the Remedy Action is not working on Windows Instance
2023-09-01 ITSI-31904 In upgrade scenario, the "Entity Discovery Searches" feature does not list the discovery search identifying entity.
2023-08-10 ITSI-31688 Episode Review share filtering url should contain the information for the episode view on/off
2023-08-10 ITSI-31706 Drill down is not working as expected in XML dashboard
2023-08-02 ITSI-31555, ITSI-31464 the ITSI integration create SNOW tickets with SPL instead of INC prefix when using Episode Action with custom endpoints with ServiceNow_TA version 7.6.0

Workaround:
Until bug in service now ADDON 7.6 bug (ADDON-64098 & ADDON-63502 ) are resolved, to avoid the issue, in ITSI, do not specify a custom endpoint in the action setup, keep the field empty.
2023-05-24 ITSI-30342 Restoring recurring and currently applicable custom threshold window is not getting activated
2023-05-10 ITSI-30068 Event Analytics Monitoring Rules Engine Information panel uses an All time search

Workaround:
  • Edit the Event Analytics Monitoring Dashboard.
  • Click on the magnifying glass under the Rules Engine Information panel.
  • Change the Time Range to Shared Time Picker (time_token).

    • 2023-04-20 ITSI-29608 itsi_bidirectional_ticking macro should use macros for index and source types being used within the search
    2023-04-20 ITSI-29609, ITSI-30886 itsi_bmc_bidirectional_ticking macro should use macros for index and source types being used within the search
    Last modified on 20 November, 2024
    Fixed issues in Splunk IT Service Intelligence   Removed features in Splunk IT Service Intelligence

    This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.17.1

    Acrobat logo Download manual
    Acrobat logo Download this page

    Was this topic useful?







    You must be logged into splunk.com in order to post comments. Log in now.

    Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

    0 out of 1000 Characters