Splunk® Enterprise

Monitoring Splunk Enterprise

Splunk Enterprise version 9.0 will no longer be supported as of June 14, 2024. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.

Resource Usage

This topic is a reference for the Resource Usage dashboards in the Monitoring Console. See About the Monitoring Console.

What do these dashboards show?

There are several Resource Usage dashboards, which you access through the Resource Usage menu. The Resource Usage: Deployment dashboard provides deployment-wide resource information, such as CPU usage, physical memory usage, and disk usage. These panels can be useful for capacity planning.

The other dashboards provide usage information by instance or machine.

Interpret results in these dashboards

About physical memory usage in these dashboards: on Linux, the OS uses free physical memory to cache filesystem resources. But memory for this is loosely bound, and the OS frees it up if a higher priority process needs it. The Monitoring Console reporting cannot discern how much memory is loosely locked up in this way.

The historical data in these dashboards comes from resource_usage.log in the _introspection index. See What data gets logged in the platform instrumentation chapter of the Troubleshooting Manual.

Resource usage: Deployment

The Deployment-Wide Median Disk Usage panel takes into account all partitions in use by each Splunk Enterprise instance.

Resource usage: Machine

This dashboard can be useful for operational post mortems, as well as for capacity planning. See the Capacity Planning Manual for more information.

In the Median CPU Usage panel, 100% means the entire system, however many cores the system has. This is in contrast to the Search Activity dashboards, where 100% means one core.

The disk space in this dashboard refers only to partitions with a Splunk Enterprise instance on them.

Resource usage: Instance

In the two "process class" panels, the value of process class can be splunkd server, search, Splunk Web, index service, scripted input, KVStore, or other.

A process class is an aggregate of processes within one class. For more information about

The index service consists of housekeeping tasks related to indexing. These tasks run at the end of the indexing pipeline but are asynchronous. These processes run on their own, not through splunkd.

The Disk Usage and Median Disk Usage panels list only the partitions that Splunk Enterprise uses.

What to look out for in these dashboards

The panels of the deployment-wide dashboard group instances by value ranges. In this dashboard, look for outliers: instances that are not like the others.

In all of the resource usage dashboards, look for patterns that appear over time. For example, in the instance and machine dashboards, look for memory usage increasing over time without recovering.

In the Instance dashboard, if a process class that is using a lot of resources turns out to be search, investigate further by going to the Search activity: Instance dashboard.

Troubleshoot these dashboards

The historical panels get data from introspection logs. If a panel is blank or missing information from non-indexers, check:

Last modified on 31 March, 2020
Search: Search Head Clustering   Resource Usage: CPU Usage

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.2.0, 9.2.1, 9.2.2

Was this topic useful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters