Splunk Cloud Platform

Splunk Cloud Platform Admin Manual

Optimize indexing and search processes

Optimizing search and indexing processes can improve your system performance and SVC utilization. Because SVC usage is based on processes performed by the search heads and indexers, optimizing these processes for efficiency can have a positive impact on your SVC usage.

However, SVC usage is not a direct measurement of the health and performance of your deployment. Improving a search or indexing process might not decrease your SVC usage but could improve your system performance. For a better understanding of your system health, see Use the Health dashboard in the Splunk Cloud Platform Admin Manual.

To learn more about SVCs, how you can monitor them using the Cloud Monitoring Console (CMC), and the workload pricing model, see the following documentation:

The following are practical tips and resources you can use to learn how to improve search and indexing processes and potentially improve SVC usage and system performance.

Optimize search processes

The following are ways you can optimize search processes so that they're more resource efficient:

Method Details
Review data models

You can use the Common Information Model (CIM) Add-on, which contains pre-configured data models that can accelerate key data. Turn on data acceleration and use CIM filters to exclude data from searches so that your searches use less resources. Make sure to include index definitions to reduce the data scanned during data model acceleration.

See the following documentation from the Common Information Model Add-on Manual:

Review skipped searches Get more details on skipped searches using the CMC Health dashboard and the CMC Skipped searches dashboard. See Investigate skipped scheduled searches and Review health indicator details in the Splunk Cloud Platform Admin Manual to learn more.

See the Splunk Blogs post Are You Skipping? Please Read and the Splunk Lantern article Reducing skipped searches to learn how you can reduce skipped searches.

Ensure scheduled searches are evenly distributed

The scheduler defers searches when there are more searches scheduled than there are available slots to run them. However, you can avoid scheduling too many searches at the same time by configuring the allow_skew setting.

If you have multiple searches that run for a few seconds at the top of the minute, you might want to set allow_skew=1m. Or if you have multiple searches that run at the top of the hour for 5 minutes, you might want to set allow_skew=45m to utilize most of the hour to run searches. See the "Schedule Skewing" section in the Splunk Blogs post Schedule Windows vs. Skewing to learn more.

Review searches that run over all time Searches that run over all time might use a lot of resources, especially if they're event searches without tokens or indexed fields that filter the data. However some searches that run over all time, such as API calls, don't use a lot of resources.
Review long time running searches and optimize SPL Improve your searches so that they're less resource intensive. Prioritize improving the most expensive searches. See Analyze expensive searches in the Splunk Cloud Platform Admin Manual and review the Expensive searches dashboard in the CMC. See About search optimization and related topics in the Splunk Cloud Platform Search Manual to learn more about optimizing your searches.
Disable unused scheduled searches Unused scheduled searches unnecessarily take up resources.
Remove unused apps and TAs Unused apps and TAs unnecessarily take up resources.

Optimize indexing processes

You can improve indexing processes by investigating data quality issues, improving data balance, and following HTTP Event Collector (HEC) best practices.

Method Details
Investigate data quality issues

Review the CMC Data Quality dashboard and see Verify data quality in the Splunk Cloud Platform Admin Manual to investigate data quality issues.

Address line breaking, event breaking, and time stamp issues to improve data quality. See the video in the Splunk Lantern article Solving data quality issues to learn more.

Improve data balance

Improve data balance to ensure your indexers are not ingesting too much or too little data by adding pipeline sets to indexers. Generally, having 2 pipeline sets per indexer is good practice. Improving data balance is especially relevant for large data sources such as syslog and firewalls, where a large amount of data is coming from one host.

Ensuring indexers are receiving a balanced amount of data will improve system performance, but might not necessarily improve aggregate SVC usage peaks. However, indexers that receive too much data might encounter issues and increase SVC usage. Balancing these indexers will improve their performance and might also improve SVC usage by resolving these issues.

Review your HTTP Event Collector (HEC) performance

To gain more insight on your HEC status, review the CMC HTTP Event Collector (HEC) dashboard and see Check the status of HTTP event collection in the Splunk Cloud Platform Admin Manual.

Last modified on 10 April, 2024
How Splunk monitors Splunk Cloud Platform   Manage Splunk Cloud Platform indexes

This documentation applies to the following versions of Splunk Cloud Platform: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308 (latest FedRAMP release), 9.1.2312

Was this topic useful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters