This chapter discusses search, the structure of a Splunk search, the search language and its syntax, some tools to help construct and troubleshoot your search, and some tips for writing better searches.
Types of searches
Before delving into the language and syntax of search, you should ask what you are trying to accomplish. Generally, after getting data into Splunk, you want to:
- Investigate to learn more about the data you just indexed or to find the root cause of an issue.
- Summarize your search results into a report, whether tabular or other visualization format.
Because of this, you might hear us refer to two types of searches: Raw event searches and Report-generating searches.
Raw event searches
Raw event searches are searches that just retrieve events from an index or indexes and are typically done when you want to analyze a problem. Some examples of these searches include: checking error codes, correlating events, investigating security issues, and analyzing failures. These searches do not usually include search commands (except
search, itself), and the results are typically a list of raw events.
- Read more about raw event searches in the "Retrieve events" chapter of this manual, beginning with About retrieving events.
Reporting searches are searches that perform some type of statistical calculation against a set of results. These are searches where you first retrieve events from an index and then pass them into one or more search commands. These searches will always require fields and at least one of a set of statistical commands. Some examples include: getting a daily count of error events, counting the number of times a specific user has logged in, or calculating the 95th percentile of field values.
- Read more about what you can do with search commands in Types of search commands.
- Read more about the structure of a search in Understanding SPL syntax.
- Read more about generating reports in the "Report on Search Results" chapter of this manual, beginning with About reporting commands.
Whether you're retrieving raw events or building a report, you should also consider whether you are running a search for sparse or dense information:
- Sparse searches are searches that look for single event or an event that occurs infrequently within a large set of data. You've probably heard these referred to as 'needle in a haystack' or "rare term" searches. Some examples of these searches include: searching for a specific and unique IP address or error code.
- Dense searches are searches that scan through and report on many events. Some examples of these searches include: counting the number of errors that occurred or finding all events from a specific host.
Search and knowledge
As you search, you may begin to recognize patterns and identify more information that could be useful as searchable fields. You can configure Splunk to recognize these new fields as you index new data or you can create new fields as you search. Whatever you learn, you can use, add, and edit this knowledge about fields, events, and transactions to your event data. This capturing of knowledge helps you to construct more efficient searches and build more detailed reports.
Search with Splunk Web, the CLI, or REST API
Most likely, you'll run a search from Splunk Web in the Search app. But, you might also run a search from the command line interface (CLI) or the REST API. Which tool is best can sometimes depend on what you want from your search.
When you search with Splunk Web, you're using the Search app, and you can control the search experience by selecting a search mode (Fast, Verbose, Smart). Depending on the mode you select, Splunk automatically discovers and extracts fields other than the default fields, returns results as an events list or a table, and runs the calculations required to generate the event timeline. Calculating the event timeline is very expensive because it creates buckets and keeps the statistics for events and fields in a dispatch directory such that it is available when the user clicks a bar on the timeline.
- Read more about how to Set search mode to adjust your search experience in this chapter.
When you run a search through the CLI or use the search jobs endpoint in the REST API to create a search, it goes directly to
splunkd without going through
splunkweb. These searches can complete much faster than the searches in Splunk Web because Splunk does not calculate or generate the event timeline. Instead, the results of a CLI search display as a raw events list or a table, depending on the type of search.
View search properties with the Search Job Inspector
About the search language
This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18