Search performance

Search performance

Splunk is optimized for text-based searching of raw event data. By default, Splunk indexes some components of each event (core fields: host, source, sourcetype). Splunk can be configured to extract and index additional components as you see fit. Performance may be affected if Splunk is:

• indexing or extracting additional fields
• accessing compressed raw data
• touching a large number of events (you can change this by altering your time range, or by setting maxresults)

You can improve Splunk's search performance by changing the segmentation, what it indexes, and how it timestamps data coming in. Here are some general guidelines to help you tune your search performance:

• Set the size of your hot db to the maximum that your system can handle. This is dependent on the amount of RAM your system contains.
• Reduce or eliminate segmentation by removing MINOR breakers, or turning some MINOR breakers into MAJOR breakers. Removing breakers is good for situations where there is a common appearing term that is separated by minor breakers. You will have to play with the breakers to optimize your searches based on the contents of the events particular to your scenario.
• Separate data into different indexes. This is an advanced technique that is only applicable if you are adding archived data while your Splunk server is indexing current data.
• Make sure that timestamping is correct on events, so that buckets capturing events don't overlap and have to be accessed all at the same time when searching.

Add-ons available on SplunkBase add-on page can also tune your search performance.

Below are some of the parameters in various configuration files that can be tuned to improve the performance of your searches.

Tuning indexes.conf

indexes.conf controls whether and how Splunk attempts to automatically learn new event types.

maxDataSize = <non-negative number> (750)

Max amount of data in MBs db hot can grow to. Tune this up to increase search performance. Values larger than the default are not recommended unless you have a 64-bit system. With a 64-bit system, it is only recommended you set this no larger than the amount of RAM in the system.

maxQueryIds = <non-negative number> (50000)

Maximum number of queryIds returned for a search. if your search tries to access more than this number of events the search will be limited to maxQueryIds. Note that this is not necessarily the same as the number of events returned, as some may be filtered out by other search terms or commands. This value is appropriate for 32-bit systems, for 64-bit you can generally specify up to 500000. Larger values may cause your instance to crash.

Tuning props.conf

props.conf controls what parameters apply to events during indexing based on settings tied to each event's source, host, or sourcetype.

DATETIME_CONFIG = <filename relative to Splunk_HOME> (/etc/datetime.xml)	Specifies the file to configure the timestamp extractor. This configuration may also be set to "NONE" to prevent the timestamp extractor from running or "CURRENT" to assign the current system time to each event.
TIME_FORMAT = <strptime-style format> (empty)	Specifies a strptime format to extract the date. Specifying a strptime format for date extraction accelerates event indexing.

Make SplunkWeb search faster

If you find that your searches are not returning quickly enough, you can make a few changes to Splunk to return results faster in SplunkWeb. Here are a few things you can try:

Enable core fields only

You can change your searches to use core fields only. Core fields include only host: source: and sourcetype::. If you do not need any additional fields for your searches, turn core fields only on by selecting the Fields drop down menu from any search results page. Check the core only (faster searching) box:

File:MakeSearchFast-Picture 2.png

Disable related

If you want to keep all your search and extracted fields, you can still increase your search time by turning off the Search for related events field action.

If you find that you are not using related events in your searches, you can use this method to vastly increase your search time.

Please note: this configuration still enables both search and extracted fields but does not support related events.

To make changes, edit SPLUNK_HOME/share/splunk/search_oxiclean/static/js/query.js.

Around line 750, you'll see the following three lines:

           if (componentClass=="resultsScrollerComponent" && !isReport) { 
               readLevel = (!isTurboOn && isVanillaSearch) ? 11 : 2; 
           } 

Changing the 11 to a 3 will permanently disable the related processing from running, and therefore enable faster search results.

Disable typeahead

Typeahead is supposed to be restricted to your current timerange. Currently, however, this only works with absolute timeranges. If you have large datasets of days, months or years, typeahead can be very slow and unduly load the server.

The short-term fix is to disable typeahead altogether. To affect this change, edit SPLUNK_HOME/share/splunk/search_oxiclean/static/js/typeahead.js

Look for the string TypeAheadQuery.prototype.send. It is the first line of a function that's about 30 lines.

In the middle of this function, you will find the following code:

   if (selectionRange.startTime) { 
       qArgs['startTime'] = selectionRange.startTime.print(window.SEARCH_TERM_TIME_FORMAT); 
   } 
   if (selectionRange.endTime) { 
       qArgs['endTime'] = selectionRange.endTime.print(window.SEARCH_TERM_TIME_FORMAT); 
   } 

Insert this new line right after the above block of code:

  if (!selectionRange.startTime || !selectionRange.endTime) return false;

That will disable typeahead entirely unless you've clicked a bar in the timeline, or are using custom time.

• Was this topic useful?
• Post a comment