Use the Health dashboard
Review the status of your Splunk Cloud Platform deployment using the Health dashboard. This dashboard provides information about the overall health of the deployment and its data collection, indexing, and search performance.
The Health dashboard data updates every 3 hours. This allows the health dashboard to load data quickly. See the Last updated column to learn when the data was last updated. The health indicator View details pages update at page load time. Because of this, there might be a discrepancy between the Health dashboard main page and the health indicator details pages.
The Health dashboard offers status information and suggested actions for health indicators so you can identify metrics that need updating or optimizing.
Review the health indicator panels
Select a health indicator panel to show indicators that affect a particular health category of your Splunk Cloud Platform deployment.
Each list item shows the corresponding indicators, the health check validation criteria, the results of the health check, and the option to configure an alert for it. The individual results data for a specific indicator correlate to the Conform, Warning, and Critical totals that display in the corresponding panel.
Selecting each panel provides details on the following health categories:
- Overall health: Provides a combined summary view of your deployment's data collection, data indexing, and data search performance in context of indicators provided in the indicator table.
- Data collection: Shows deployment's universal forwarders and heavy forwarders days remaining before expiration.
- Data indexing: Shows the current state of bucket size and range per index for your deployment.
- Data search: Shows the current state of skipped search percentage, high memory searches, and cache transfer activity in your deployment.
Review health indicator toggled summary view
Select the toggle next to an indicator to view a description of what the indicator evaluates, when the indicator is marked as warning or critical, and suggested actions to maintain the health of the indicator.
Configure alerts for health indicators
CMC includes preconfigured alerts so you can get informed when health indicators reach certain thresholds. Select Configure to enable an alert. The following table describes the threshold for each health indicator preconfigured alert:
Health indicator | Alert trigger |
---|---|
Universal forwarder software version | When your universal forwarder is going to expire within 15 days. |
Heavy forwarder software version | When your universal forwarder is going to expire within 15 days. |
Bucket size and range | When your bucket size is not well distributed. |
Skipped search percentage | When your skipped search percentage is greater than 25%. |
Cache transfer activity | When your SmartStore download size exceeds 10% of total disk space. |
High memory searches | When your searches are consuming more than 10% of Splunk Cloud Platform instance memory. |
Review health indicator details
In the toggled expanded view, select View details for any of the health indicators to view a drilldown of the indicator. Select a status card to filter the list by Conforming, Warning, or Critical status. The status column correlates to the Conforming, Warning, and Critical totals that display in top status cards.
The detailed view for each health indicator displays the following information:
Health indicator | Chart description | Health indicator importance |
---|---|---|
Universal forwarder software version |
The Universal forwarder software version detailed views show the forwarder names, versions, days to expiration, expiration timestamp, and status. |
This indicator informs you of upcoming expiry dates of your universal forwarder software version so you can maintain version compatibility with Splunk Cloud Platform. Maintaining version compatibility allows you to immediately take advantage of new capabilities and ensures uninterrupted service.
|
Heavy forwarder software version |
The Heavy forwarder software version detailed view shows the heavy forwarder names, versions, days to expiration, expiration timestamp, and status. |
This indicator informs you of upcoming expiry dates of your heavy forwarder so you can maintain version compatibility with Splunk Cloud Platform. Maintaining version compatibility allows you to immediately take advantage of new capabilities and ensures uninterrupted service.
|
Bucket size and range | The Bucket size and range detailed view shows the index, bucket type, caller, quarantined percentage, full percentage, exceeded count, small percentage, small count, total count, and status for each bucket |
This indicator evaluates buckets and their size in an index to help you manage optimal bucket sizes. If bucket sizes fall below or exceed the range 375MB to 750MB, your stack might experience degraded performance from excessive cache calls. If buckets frequently exceed their maximum configured sign, it might be due to insufficient indexing capacity. Select a critical status index to view more information about source types and other bucket details. A longer period of time between event time and processing time might indicate a need for optimization. |
Skipped search percentage | The Skipped search percentage detailed view shows the app, saved search, user, skip ratio, percentage skipped, reason, and status. | This indicator evaluates the skipped search ratio of all scheduled searches. A high ratio of skipped scheduled searches might indicate one of the following causes:
|
Cache transfer activity | The Cache transfer activity detailed view shows the index, download amount, cache churn percentage, and status. | This indicator evaluates cache download size per index and informs you when data downloaded from SmartStore exceeds 10% of total disk space. Keeping cache download size below 5% ensures proper use of infrastructure resources and prevents unnecessary cache churn that slows down searches. |
High memory searches | The High memory searches detailed view shows the search IDs, memory used, percentage memory used, and status.
The High memory searches detailed view returns the first 50,000 searches sorted by critical, warning, then conforming status. This prevents the results from timing out on large stacks. |
This indicator evaluates search size and informs you when searches take up a large amount of memory. High memory searches might cause your Splunk Cloud Platform instance to not function if it runs out of memory. |
Get recommended actions based on status
The Health dashboard includes help panels that provide recommended actions and information that can help you proactively update expiring forwarders, improve search queries and search time, and maintain well-distributed buckets.
The following health indicators include help panels:
- Universal forwarder software version
- Heavy forwarder software version
- High memory searches
- Skipped search percentage
- Bucket size and range
- Cache transfer activity
To view a help panel, navigate to the indicator dropdown > View details > any list item. The panel provides status information, recommended action, and additional information about maintaining the health of the selected indicator.
Health indicator information and additional resources
The following table provides information on each health indicator, what the health indicator informs you of, and additional resources for further knowledge and troubleshooting:
Health indicator | Description | Additional resources |
---|---|---|
Universal forwarder software version |
The universal forwarder streams data from your machine to a data receiver, formats data before sending it to your Splunk platform, and lets you monitor data in real time. Maintaining version forwarder version compatibility ensures there is no interruption to your service. This indicator evaluates the software version for all universal forwarders and informs you of upcoming expiry dates so you can maintain version compatibility. |
To learn more about the universal forwarder, see About the universal forwarder in the Splunk Universal Forwarder Forwarder Manual. For details on upgrading your universal forwarder, see Upgrade your Forwarder in the Splunk Cloud Platform Admin Manual. For more information on forwarder version compatibility, see Monitor forwarder deployments in the Splunk Cloud Platform Admin Manual and Supported forwarder versions in the Splunk Cloud Platform Service Description. |
Heavy forwarder software version |
A heavy forwarder parses data before forwarding or indexes data locally while forwarding the data to another indexer. This indicator evaluates the software version for all heavy forwarders and informs you of upcoming expiry dates so you can maintain version compatibility. |
To learn more about the heavy forwarder, see Heavy and light forwarders in the Splunk Enterprise Forwarding Data manual. For details on upgrading your heavy forwarder, see Upgrade your Forwarder in the Splunk Cloud Platform Admin Manual. For more information on forwarder version compatibility, see Monitor forwarder deployments in the Splunk Cloud Platform Admin Manual and Supported forwarder versions in the Splunk Cloud Platform Service Description. |
High memory searches |
High memory searches use a significant amount of your Splunk platform instance memory. This indicator evaluates search size and informs you of when searches take up a high amount of memory. |
See the Expensive searches dashboard in the CMC for more details about searches that are using a lot of memory. To learn more about how to troubleshoot high memory, see TLimit search process memory usage in the Splunk Cloud Platform Search Manual. For information on how to write more efficient searches, see Write better searches in the Splunk Cloud Platform Search Manual. |
Bucket size and range |
An index typically consists of buckets, directories that contain processed external data and index files. This indicator evaluates buckets and their size in an index to help you manage optimal bucket sizes. |
To learn more about buckets and how they work in an index, see How the indexer stores indexes in the Splunk Enterprise Managing Indexers and Clusters of Indexers manual. |
Skipped search percentage |
Skipped searches occur when the load on your system is higher than the available resources. This indicator evaluates the skipped search ratio of all scheduled searches. |
To learn more about why searches are being skipped and how to avoid skipped searches, see Are you Skipping? Please read!. To investigate skipped scheduled searches, use the Search > Skipped scheduled searches dashboard. See the documentation at Investigate skipped scheduled searches to learn how to review this dashboard. |
Cache transfer activity |
Cache transfer activity refers to your deployment's local storage, aggregated by bucket size. This indicator evaluates bucket download size and informs you of high download size so you can optimize your deployment's cache transfer activity. |
To investigate cache transfer activity further, use the Search > Search usage statistics dashboard. |
Provide feedback
You can provide feedback and ask questions directly from the Health dashboard. Select the Feedback button to ask the Splunk community a question or submit an idea for the Splunk team.
Use the Overview dashboard | Use the Maintenance dashboard |
This documentation applies to the following versions of Splunk Cloud Platform™: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release), 9.3.2408
Feedback submitted, thanks!