Quick tips for optimization

The key to fast searching is to limit the data that needs to be pulled from disk to an absolute minimum. Filter the data as early as possible in the search, so that processing is done on the minimum amount of data necessary.

Limit the data from disk

The techniques to limit the amount of data retrieved from disk range from setting a narrow time window, being as specific as possible, and retrieving the smallest number of events necessary.

Narrow the time window

One of the most effective ways to limit the data that is pulled off from disk is to limit the time range. Use the time range picker or specify time modifiers in your search to identify the smallest window of time necessary for your search.

If you need to see data from only the last hour, do not use the default time range of Last 24 hours.

• See Select time ranges to apply to your search
• See Specify time modifiers in your search

If you must use a broad time range, such as Last week or All time, then use other techniques to limit the amount of data retrieved from disk.

Specify the index, source, or source type

Understanding how your data is organized is important to optimizing your searches. Take the time to learn which indexes contain your data, the sources of your data, and the source types. Knowing this information about your data helps you narrow down your searches.

1. Run the following search.

   source=*

   This search not optimized, but it does provide an opportunity for you to learn about the data you have access to.

2. In the Selected fields list, click on each type of field and look at the values for host, source, and sourcetype.
3. In the Interesting fields list, click on the index field. Look at the names of the indexes that you have access to.

Whenever possible, specify the index, source, or source type in your search. When Splunk software indexes data, it automatically tags each event with a number of fields. The index, source, and source type fields are added automatically to each event as default fields. A default field is an indexed field that the Splunk software recognizes in your event data at search time. The host, source, and source type fields describe where the event originated.

Be specific

Use the most specific terms in your search that you can. If possible, avoid using wildcard characters.

For example, instead of using a wildcard character for a keyword:

*error

Use the specific keyword:

fatal_error

Here is another example.

Instead of using a wildcard character for field values:

status=404 OR status=5*

Specify each value:

status=404 OR status=500 OR status=503

Combine a source type or an index with one or more field-value pairs. For example:

sourcetype=access_* status=200 action=purchase

This search retrieves events from only your web access logs. A wildcard character, access_*, is used in the field value to match any Apache web access source type. The source types can be access_common, access_combined, or access_combined_wcookie. Two specific field-value pairs are included in the search, status=200 and action=purchase.

Limit the number of events retrieved

By default, a Splunk search retrieves all events. However in some situations you might want to retrieve a sample set of events, instead of retrieving the entire event set. Limiting the number of events retrieved is useful in several situations:

• You are creating a search and want to determine if you are retrieving the correct events
• You need only a subset or sample set of events for your search

You can specify a limit to the number of events retrieved in a couple of ways:

Use the head command

The head command retrieves only the most recent N events for a historical search, or the first N captured events for a realtime search. For example:

sourcetype=access_* | head 1000 ...

Use event sampling

Event sampling uses a ratio that you specify to select events. For example, if the sample ratio value is 100, each event has a 1 in 100 chance of being included in the result set. To learn more about event sampling and sampling ratios, see Event sampling.

By default, event sampling is not active. You must specify a sampling ratio before you run your search. In Splunk Web, click the Sampling drop-down and choose a sampling ratio.

Use the TERM directive to match terms that contain minor breakers

The TERM directive is useful when you are searching for a term that contains minor breakers, such as periods or underscores, but does not contain major breakers. The term must be bound by major breakers, such as spaces or commas.

For example, the IP address 192.0.2.255 contains the period ( . ) minor breaker. If you specify TERM(192.0.2.255), the Splunk software treats the IP address as a single term, instead of individual numbers.

Using the TERM directive to search for terms that contain minor breakers improves search performance. See Event segmentation and searching.

Avoid using NOT expressions

More resources are used tracking NOT expressions than if you specify what you are looking for. Where ever possible, avoid using NOT expressions. For example, instead of using a string of NOT or != expressions such as:

(NOT host=d NOT host=e)

or

(host!=d AND host!=e)

Use the specific terms you are searching for:

(host=a OR host=b OR host=c).

To learn more, see Difference between NOT and !=.

Filter as soon as possible

Filter results as soon as possible before performing calculations. You can use field-value pairs and commands to filter results.

Use field-value pairs before the first pipe

Field-value pairs are indexed. Specifying field-value pairs before the first pipe is an efficient way to filter out events.

For example, in the following search the term status=404 is in a separate search:

ERROR | search status=404

Move the term status=404 before the first pipe:

ERROR status=404

Here is another example.

The second search includes the term clientip="10.0.0.0/8". There is no reason to wait to filter on that term.

ERROR | stats sum(bytes) as sum by clientip | search sum >1048576 AND clientip="10.0.0.0/8"

Move the term clientip="10.0.0.0/8" to filter out all other clientip addresses before the stats command.

ERROR clientip="10.0.0.0/8" | stats sum(bytes) by clientip | search sum > 1048576

Use filtering commands before calculating commands

Use filtering commands, such as where, before commands that perform calculations, such as eval.

For example, this search has a where command after the eval command. The search does not require the results of the eval command before the where command is run.

field1=value | eval KB=bytes/1024 | where field2=field3

Move the where command to filter the results before the eval command is processed:

field1=value | where field2=field3 | eval KB=bytes/1024

Filter unnecessary fields from search results

You can remove unnecessary fields from the search results by using commands such as fields. Removing extraneous fields from searches make them more efficient because search doesn't have to process superfluous data. Other search optimizations that work on a definite set of fields, such as predicate pushdown and transforming commands, already reduce the number of fields that are propagated and, therefore, can make the fields command less impactful.

Use non-streaming commands as late as possible

Postpone commands like sort and stats as late as possible in your search. These commands are referred to as non-streaming commands. Before these commands can run, the entire result set must be returned. For example, the results cannot be sorted until all of the results are available.

For an explanation about the differences between streaming and non-streaming commands, see Types of commands.

For a list of of commands by type, see Command types in the Search Reference.

Other techniques for search optimization

There are a few other techniques that you can use to optimize your searches.

• Store your apps on a fast, local disk, not on network file system (NFS). Loading apps on NFS can become a performance bottleneck.

• Use post-process searches in dashboards. See Searches power dashboards and forms in Dashboards and Visualizations.

• Use summary indexing, report acceleration, and data model acceleration features.

• Use Fast Mode to increase the speed of searches by reducing the event data that the search returns. See Search modes.

See also

Related information

About search optimization

Write better searches

Built-in optimizations