Search Manual

Quick tips for optimization

The key to fast searching is to:

1. Limit the data that needs to be pulled off from disk to an absolute minimum.

2. Filter the data as early as possible in the search, so that processing is done on the minimum data necessary

Limit the data from disk

One of the most effective ways to limit the data that is pulled off from disk is to limit the time range to only what is absolutely needed.

For example -1h not -1w, or earliest=-1d.

Learn more about specifying time ranges in About time ranges.

Another way to limit the data that is pulled off from disk is to partition data into separate indexes. If you rarely search across more than one type of data at a time, partition your different types of data into separate indexes. Then restrict your searches to the specific index. For example, store Web access data in one index and firewall data in another. This is recommended for sparse data, which might otherwise be buried in a large volume of unrelated data.

Read more about "Ways to set up multiple indexes" in the Managing Indexers and Clusters of Indexers manual and how to "Retrieve events from indexes" in this manual.

• Use Fast Mode to increase the speed of searches by reducing the event data that they return.
• Search as specifically as you can. For example, fatal_error not *error*
• Filter out results as soon as possible before calculations. Use field-value pairs, before the first pipe. For example, ERROR status=404 |… instead of ERROR | search status=404… Or use filtering commands such as where.
• Filter out unnecessary fields as soon as possible in the search.
• Postpone commands that process over the entire result set (non-streaming commands) as late as possible in your search. Some of these commands are: dedup, sort, and stats.
• Use post-processing searches in dashboards.
• Use summary indexing, report acceleration, and data model acceleration features.