Splunk® IT Service Intelligence


This documentation does not apply to the most recent version of Splunk® IT Service Intelligence. For documentation on the most recent version, go to the latest release.

Application Server Module troubleshooting

Verify Data Collection

Verify that the add-ons in your deployment are installed and configured correctly by checking the add-on's indices, sources or source types.

Add-on Data verification search
Tomcat index = * sourcetype = tomcat:*
Websphere index = * sourcetype = index=* sourcetype = ibm:was*

Data population can also be verified by checking to see if your tags have data.

Tag Search
appserver tag=appserver
inventory tag=appserver tag=inventory
inventory-->application tag=appserver tag=inventory tag=application
performance tag=appserver tag=performance
performance-->transactions tag=appserver tag=performance tag=transactions
performance-->runtime tag=appserver tag=performance tag=runtime
performance-->throughput tag=appserver tag=performance tag=throughput
performance-->session tag=appserver tag=performance tag=session

Verify inventory data

Index Search Data Model Search
|search application_server=*
|table application_server,role,vendor_product,version_number,vendor_product_runtime,vendor_product_runtime_version,full_host_name
|dedup application_server
|tstats latest(Inventory.ip_address) as ip_address,
latest(Inventory.product_family) as product_family,
latest(Inventory.vendor_product) as vendor_product,
latest(Inventory.version_number) as version_number,
latest(Inventory.vendor_product_runtime) as vendor_product_runtime,
latest(Inventory.runtime_vendor) as runtime_vendor,
latest(Inventory.vendor_product_runtime_version) as vendor_product_runtime_version,
latest(Inventory.appserver_port_number) as appserver_port_number,
latest(Inventory.root_role) as role,
latest(Inventory.full_host_name) as full_host_name,
latest(Inventory.application_server) as application_server
from datamodel=AppServer.Inventory
where Inventory.ip_address=* 
by host
| fields + application_server,role,vendor_product,version_number,vendor_product_runtime,vendor_product_runtime_version,full_host_name

Verify KPI data

Search Expected Results
| timechart avg(active_sessions_count) as "Active Sessions (count)"
Numbers populate on the Statistics tab with the field Active Sessions(count)
| timechart avg(cpu_utilization_percent) as cpu_util_pct
| eventstats avg(cpu_util_pct) as average
| eventstats min(cpu_util_pct) as minimum
| eventstats max(cpu_util_pct) as maximum
| eval _stats = "Range: ".round(minimum, 1)." - ".round(maximum, 1)."%, "."Average: ".round(average, 1)."%"
| table cpu_util_pct,average, minimum, maximum
Numbers populate on the Statistics tab in a table with columns cpu_util_pct, average, minimum, and maximum

Entities not found

When you are unable to locate operating system entities, see the ITSI module troubleshooting section on entities.

Troubleshooting entity appearance speed

It will take about an hour to start seeing a full sparkline chart and for KPIs to become active. Learn about entity appearance speed.

Metrics not collected

If you are unable to view your data, run the data model audit to make sure your data models are processing your data.

If you are running into issues seeing your KPI data, see ITSI module troubleshooting section on KPIs.

User Permissions

If a user encounters a permission-related obstacle, the issue could be related to their assigned role.

ITSI permissions are determined by the role that each user has. Each role offers a different set of user permissions.

See ITSI module troubleshooting to learn about permissions within ITSI.

Learn about adding navigation to a Splunk app.

The Drilldown page stops functioning

If the drilldown page stops working while using ITSI:

  1. Launch a web browser and
  2. Use your console to debug the JavaScripts,
  3. Examine the results to see if any KPI templates are corrupted.

Tomcat lookup field error

If you have both the Tomcat and WebSphere add-ons installed on the same server, and encounter the following error about Tomcat.

Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'tomcat:access:log' and lookup table 'tomcat_rootdir_jmxhost_lookup'.
Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'tomcat:runtime:log' and lookup table 'tomcat_rootdir_jmxhost_lookup'. [tag-asnixdev3.sv.splunk.com]
Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'tomcat:access:log' and lookup table 'tomcat_rootdir_jmxhost_lookup'. [tag-asnixdev3.sv.splunk.com]
Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'tomcat:runtime:log' and lookup table 'tomcat_rootdir_jmxhost_lookup'.

Check the $splunk/etc/apps/Splunk_TA_ibm-was/lookups/ folder to see if the file ibm_was_inventory.csv is present. If it is not there, open the WebSphere add-on installation package, and copy the ibm_was_inventory.csv file into the $splunk/etc/apps/Splunk_TA_ibm-was/lookups/ folder.

Last modified on 28 April, 2023
Application Server Module data model reference table   About the Database Module

This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.11.0, 4.11.1, 4.11.2, 4.11.3, 4.11.4, 4.11.5, 4.11.6, 4.12.0 Cloud only, 4.12.1 Cloud only, 4.12.2 Cloud only, 4.13.0, 4.13.1, 4.13.2, 4.13.3, 4.14.0 Cloud only, 4.14.1 Cloud only, 4.14.2 Cloud only, 4.15.0, 4.15.1, 4.15.2, 4.15.3, 4.16.0 Cloud only, 4.17.0, 4.17.1, 4.18.0, 4.18.1

Was this topic useful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters