Optimize indexing and search processes

Optimizing search and indexing processes can improve your system performance and SVC utilization. Because SVC usage is based on processes performed by the search heads and indexers, optimizing these processes for efficiency can have a positive impact on your SVC usage.

However, SVC usage is not a direct measurement of the health and performance of your deployment. Improving a search or indexing process might not decrease your SVC usage but could improve your system performance. For a better understanding of your system health, see Use the Health dashboard in the Splunk Cloud Platform Admin Manual.

To learn more about SVCs, how you can monitor them using the Cloud Monitoring Console (CMC), and the workload pricing model, see the following documentation:

• Monitor current SVC usage of your workload-based subscription in the Splunk Cloud Platform Admin Manual.
• Performance considerations in the Splunk Cloud Platform Service Description.

The following are practical tips and resources you can use to learn how to improve search and indexing processes and potentially improve SVC usage and system performance.

Optimize search processes

The following are ways you can optimize search processes so that they're more resource efficient:

Method	Details
Review data models	You can use the Common Information Model (CIM) Add-on, which contains pre-configured data models that can accelerate key data. Turn on data acceleration and use CIM filters to exclude data from searches so that your searches use less resources. Make sure to include index definitions to reduce the data scanned during data model acceleration. See the following documentation from the Common Information Model Add-on Manual: Overview of the Splunk Common Information Model Accelerate CIM data models Use the CIM Filters to exclude data
Review skipped searches	Get more details on skipped searches using the CMC Health dashboard and the CMC Skipped searches dashboard. See Investigate skipped scheduled searches and Review health indicator details in the Splunk Cloud Platform Admin Manual to learn more. See the Splunk Blogs post Are You Skipping? Please Read and the Splunk Lantern article Reducing skipped searches to learn how you can reduce skipped searches.
Ensure scheduled searches are evenly distributed	The scheduler defers searches when there are more searches scheduled than there are available slots to run them. However, you can avoid scheduling too many searches at the same time by configuring the allow_skew setting. If you have multiple searches that run for a few seconds at the top of the minute, you might want to set allow_skew=1m. Or if you have multiple searches that run at the top of the hour for 5 minutes, you might want to set allow_skew=45m to utilize most of the hour to run searches. See the "Schedule Skewing" section in the Splunk Blogs post Schedule Windows vs. Skewing to learn more.
Review searches that run over all time	Searches that run over all time might use a lot of resources, especially if they're event searches without tokens or indexed fields that filter the data. However some searches that run over all time, such as API calls, don't use a lot of resources.
Review long time running searches and optimize SPL	Improve your searches so that they're less resource intensive. Prioritize improving the most expensive searches. See Analyze expensive searches in the Splunk Cloud Platform Admin Manual and review the Expensive searches dashboard in the CMC. See About search optimization and related topics in the Splunk Cloud Platform Search Manual to learn more about optimizing your searches.
Disable unused scheduled searches	Unused scheduled searches unnecessarily take up resources.
Remove unused apps and TAs	Unused apps and TAs unnecessarily take up resources.

Optimize indexing processes

You can improve indexing processes by investigating data quality issues, improving data balance, and following HTTP Event Collector (HEC) best practices.

Method	Details
Investigate data quality issues	Review the CMC Data Quality dashboard and see Verify data quality in the Splunk Cloud Platform Admin Manual to investigate data quality issues. Address line breaking, event breaking, and time stamp issues to improve data quality. See the video in the Splunk Lantern article Solving data quality issues to learn more.
Improve data balance	Improve data balance to ensure your indexers are not ingesting too much or too little data by adding pipeline sets to indexers. Generally, having 2 pipeline sets per indexer is good practice. Improving data balance is especially relevant for large data sources such as syslog and firewalls, where a large amount of data is coming from one host. Ensuring indexers are receiving a balanced amount of data will improve system performance, but might not necessarily improve aggregate SVC usage peaks. However, indexers that receive too much data might encounter issues and increase SVC usage. Balancing these indexers will improve their performance and might also improve SVC usage by resolving these issues.
Review your HTTP Event Collector (HEC) performance	To gain more insight on your HEC status, review the CMC HTTP Event Collector (HEC) dashboard and see Check the status of HTTP event collection in the Splunk Cloud Platform Admin Manual.