Splunk® Enterprise

Monitoring Splunk Enterprise

Splunk Enterprise version 7.2 is no longer supported as of April 30, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Indexing performance dashboards

This topic is a reference for the Indexing performance dashboards in the Monitoring Console. See About the Monitoring Console.

What do these dashboards show?

The Indexing performance: Deployment dashboard provides an overview of indexing performance across your Splunk Enterprise deployment.

The Indexing Performance: Instance dashboard contains panels about indexing performance on one instance of the potentially several in your deployment. These panels can provide more detailed insight into problems that surface in the deployment-wide dashboard.

Interpret results in these dashboards

Indexing performance: Deployment

In the Overview of Indexing Performance panel, total indexing rate is summed over all indexers.

In the Instances by Estimated Indexing Rate panel, the indexing rate is estimated because it uses metrics.log, which takes only the top ten results for each type by default. See About metrics.log in the Troubleshooting Manual.

Indexing performance: Instance

The snapshot panel called Splunk Enterprise Data Pipeline exposes decaying averages for queue sizes. The averages use data over the previous 15 minutes. This panel, along with the historical panel Median Fill Ratio of Data Processing Queues, helps you narrow down sources of indexing latency to a specific queue. Data starts at parsing and travels through the data pipeline to indexing at the end.

The Aggregate CPU Seconds Spent per Indexer Processor Activity panel lets you "Split index service by subtask." The several index services are subtasks related to preparing for and cleaning up after indexing. For more information about the subtask categories, see the metrics.log topic in the Troubleshooting Manual.

What to look out for in these dashboards

In the Indexing performance: Instance dashboard, the Splunk Enterprise Data Pipeline panel, along with the historical panel Median Fill Ratio of Data Processing Queues, helps you narrow down sources of indexing latency to a specific queue. Data starts at parsing and travels through the data pipeline to indexing at the end. Here is an example of the panel in an instance with unhealthy queues:

Cloggedpipeline.png

In this example, although the parsing and aggregator queues have very high fill ratios, the problem is likely to be with processes in the typing queue. The typing queue is the first one that slows down, and data is backing up into the other two queues while waiting to get into the typing queue.

For more information about troubleshooting indexing performance problems, see Identify and triage indexing performance problems in the Troubleshooting Manual.

Troubleshoot these dashboards

The snapshot panels get data from Splunk REST endpoints for introspection. If snapshot panels lack data, check

  • the system requirements for platform instrumentation.
  • the pipelinesets setting in server.conf. When pipeline sets are used (that is, if pipelinesets is set to a value greater than 1), some panels of the Monitoring Console indexing performance dashboards will be blank.

The historical panels for these dashboards get data from metrics.log.

Last modified on 25 February, 2019
Access and customize health check   Indexing: Indexes and Volumes

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters