Search performance

Search performance

Splunk is optimized for text-based searching of raw event data. By default, Splunk indexes some components of each event (default fields: host, source, sourcetype). Splunk can be configured to extract and index additional components as you see fit. Performance may be affected if Splunk is:

• Indexing or extracting additional fields.
• Accessing compressed raw data.
• Accessing a large number of events (you can change this by altering your time range, or maximum results you search for).

You can improve Splunk's search performance by changing indexing properties such as time stamping and segmentation. Here are some general guidelines to help you tune your search performance:

• Set the size of your hot db to the maximum size that your system can support. This is dependent on the amount of RAM your system contains.
• Reduce or eliminate segmentation by removing MINOR breakers, or turning some MINOR breakers into MAJOR breakers. Play with the breakers to optimize your searches based on the contents of the events particular to your scenario.
• Separate data into different indexes. This is an advanced technique that is only applicable if you are adding archived data while your Splunk server is indexing current data.
• Make sure that time stamping is correct on events.

Below are some of the parameters in various configuration files that may improve your search performance.

Configure system memory access

Determine how Splunk accesses system memory via indexes.conf.

maxDataSize = <non-negative number>

The maximum size in MBs of the hot DB. The hot DB will grow to this size before it is rolled out to warm. Defaults to 750 on a 32-bit system, 10000 on a 64-bit system. Do not change these values unless specifically advised to do so by a Splunk Engineer.

Configure indexing properties

Configure indexing properties via props.conf. Control indexing properties based on settings tied to each event's source, host, or source type.

DATETIME_CONFIG = <filename relative to Splunk_HOME>	Specifies the file to configure the timestamp extractor. This configuration may also be set to "NONE" to prevent the timestamp extractor from running or "CURRENT" to assign the current system time to each event. Defaults to /etc/datetime.xml (eg $SPLUNK_HOME/etc/datetime.xml).
TIME_FORMAT = <strptime-style format> (empty)	Specifies a strptime format to extract the date. Specifying a strptime format for date extraction accelerates event indexing.

Configure Splunk Web settings

Configure many of Splunk Web's settings via web.conf. You can configure the following attributes to make searching faster.

numberOfEventsPerCard = <integer>	Configuration for the number of events that the Endless Scroller asks the server for with each request. Defaults to 10.
numberOfCardsPerDeck = <integer>	Configuration for the number of requests that the Endless Scroller will make before it starts to recycle space occupied by prior pages. Defaults to 7.

Configure indexed fields

In some situations, you can increase search performance by extracting fields at index time. Review the documentation on creating indexed fields, particularly the Note regarding performance to determine whether they are likely to help in your environment.

Configure Splunk Web

You can increase search performance by changing various configuration settings in the Preferences menu of Splunk Web.

Disable typeahead

Typeahead is not restricted to your current time range. If you have large datasets of days, months or years, typeahead can be very slow and load the server. This can be especially problematic in a distributed search environment.

You can disable typeahead altogether using a role capability in authorize.conf.

By default the typeahead capability is added to the User role in etc/system/default/authorize.conf, and is inherited by the Power and Admin roles. Thus, disabling it for the user role will disable it for all roles, and all users.

In $SPLUNK_HOME/etc/system/local/authorize.conf add the following settings.

[role_User]
get_typeahead = disabled

If you have a different role scheme, you will have to interpret these instructions within that scheme.

Set segmentation in Splunk Web

Change segmentation settings in the Preferences tab in Splunk Web. For example, raw segmentation produces faster searching, but doesn't give you the ability to add search terms to you search by clicking on parts of any event. Play around with the different segmentation settings to find which one is the best for your data.

• Was this topic useful?
• Post a comment