Searches power dashboards and forms
Searches generate visualizations and other content in dashboards and forms. You can also use searches to implement dynamic or interactive behavior in a dashboard.
Learn about the available search types and how to work with them in Simple XML.
Overview of search types and uses
You can use one or more search types in a dashboard depending on the content or behavior that you are creating.
Inline searches
Inline searches exist directly in a dashboard or visualization. You can create or modify inline searches when you edit dashboard panels.
Each visualization can have its own inline search. You can also use an inline search as a base search in a dashboard. Base searches require post-process searches to modify results and generate visualizations in dashboard panels.
Inline searches run in "fast mode" which means that not all fields may be extracted. To ensure that all fields are extracted you can do one of the following:
- Add
to your search.<Your_search> | fields * | ...
- Add
to your search.fieldname=*
Referenced report searches
Use a search saved as a report to generate dashboard content. Use a reference to the report to add its search and visualization to a dashboard or form.
You can edit the dashboard to adjust the visualization and time range from a referenced report. However, you cannot edit the search directly in the dashboard. Edit the report to change the search.
When you change a report search, referenced report searches in dashboards update automatically.
Searches to populate form inputs
You can use a search to generate form input choices dynamically. Indicate a results field to use for the choice labels and values that users see in the form.
Post-process searches
Post-process searches perform additional processing on results from a base search. You can use multiple post-process searches to generate different results from the same base search. You can also populate form inputs with a post-process search. For more details, see the best practices for post-process searches.
Prebuilt panels
Create a prebuilt panel if you want to reuse a search and other content. To change a prebuilt panel search, edit the prebuilt panel directly.
Changes to a prebuilt panel appear automatically in dashboards where the panel is used.
Searches generated with Pivot
You can generate pivots for export to dashboards. For more information, see Design pivot tables with the Pivot Editor in the Pivot Manual.
Create searches in Simple XML
The <search>
element defines a search in Simple XML source code. Search elements include child elements, such as <query>
for the search string and elements for the time range.
You can use a <search>
element to define searches generating dashboard or form content. You can also use a <search>
to generate form input choices or define post-process searches.
See the Simple XML Reference for more details on the the <search>
element, its child elements, and usage requirements.
If you have multiple saved searches in your dashboard, your dashboard only runs as many searches as your concurrency limit. Your limits.conf
file lists concurrency limits for different types of searches. For example, if your concurrency limit is three and you have five saved searches, the first three saved searches in the queue run, but the last two don't. Concurrency limits prevent scheduled searches from queueing indefinitely.
For dashboards with more saved searches than the concurrency limit, use an ad hoc or inline search with the savedsearch
command to call the saved search. An ad hoc or inline search creates a scenario where searches that exceed the concurrency limit still queue and run when capacity allows.
For a list of settings for search concurrency limits, see Concurrency in the Admin Manual.
Examples
Referenced search from a report
Use and modify a referenced saved search to generate dashboard panel content.
Using tokens to add dynamic values to searches
Tokens are references to dynamic data, such as search result counts or user specified values. Use tokens within a search to access dynamic values and generate more customized results.
To use a token in a search, use $...$
delimiter characters around the token name. When the search runs, the $<token_name>$
is replaced with the dynamic value that it references.
Example
This search string includes the $series_tok$
token to represent a dynamic series
field value. This value might come from a user selection in a form input.
index=_internal source=*metrics.log group="per_sourcetype_thruput" series=$series_tok$ | table sourcetype eps, kb, kbps
The search can generate a visualization representing the user selected $series_tok$
value.
In addition to searches, tokens can also be used to capture user input values and control dynamic dashboard behavior. See Token usage in dashboards for more details.
Post-process searches
Post-process searches perform additional processing on results from a base search. A base search can be a global search or any other search within a dashboard. Use the base
attribute in a post-process <search>
to indicate the base search id.
You can use a single post-process search to generate results or you can chain multiple post-process searches together.
Best practices
Use these best practices to make sure that post-process searches work as expected.
Use a transforming base search
A base search should be a transforming search that returns results formatted as a statistics table.
Non-transforming base searches can cause the following search result and timeout issues. If you observe these issues in a dashboard, check the base search to make sure that it is a transforming search.
No results returned
If the base search is a non-transforming search, you must explicitly state in the base search what fields will be used in the post-process search using the | fields
command. For example, if your post-process search will search for the top selling buttercup game categories over time, you would use a search command similar to the following.
| fields _time, categoryId, action
Event retention
If the base search is a non-transforming search, the Splunk platform retains only the first 500,000 events that it returns. A post-process search does not process events in excess of this 500,000 event limit, silently ignoring them. This can generate incomplete data for the post-process search.
This search result retention limit matches the max_count
setting in limits.conf
. The setting defaults to 500,000.
Client timeout
If the post-processing operation takes too long, it can exceed the Splunk Web client timeout value of 30 seconds.
collect
The collect
command does not work with post-process searches when used in the base search. Use an inline search instead of a base search to use the collect
command.
For more information about transforming searches, see transforming commands and searches in the Search Manual.
Do not reference fields in post-process searches that are not referenced in the base search
A post-process search depends entirely on the fields present in the base search. If you are not referencing a particular field in the base search, do not reference it in the post-process search. Fields used in transforming commands will automatically be available for post-process searches. When transforming commands are not used in a base search, fields without a reference in the base search appear null in a post-process search. The post-process search returns no results in this case.
Utilize fewer base searches
Using fewer base searches can improve your dashboard's performance. For example, one base search is often more efficient than multiple base searches.
Limit base search results and post-process complexity
Passing a large number of search results to a post-process search can cause server timeout issues. In this scenario, consider adjusting the base search to reduce the number of results and fields that it returns. You can also consider reducing the complexity of post-process operations on the base search results.
Adding comments in a chain search
Comments in chain searches have a particular behavior when handling pipes. By default, the Splunk Platform adds a pipe to the beginning of each chain search unless a user initially includes the pipe. When adding a comment, remove the pipe from the start of the chain search or include the comment after the first pipe of the chain search.
Best practices for using comments with chain searches:
The following is an example of not including a pipe at the start of the chain search:
```comment``` chart count
This is an example of how to include the comment after the first pipe of the chain search:
| ```comment``` chart count
Examples
The following examples show you how to use base searches and post-process searches.
Basic post-process search
The base search in this example uses the <stats>
transforming command. Two post-process searches use the base search results differently.
Chained post-process searches
Chain two or more post-process searches together. Start with a base search and post-process search. Use the first post-process search as the base search for another post-process search.
Complex post-process search
When creating a complex base search that includes statistical aggregations such as percentiles or standard deviations, use summary indexing commands.
Post-process search to populate form inputs
Use a post-process search to populate a form input
Panel search controls
Click refresh and stop buttons on a dashboard panel to generate new search results or stop a running search job. The stop button replaces the refresh button in the panel while a search is running. Visualizations render available results from stopped search jobs and a timestamp shows the time of the most recent job.
By default, the stop search button is visible. You can hide this button using the link.stopSearch.visible
Simple XML option in your dashboard source code. See the Simple XML Reference for more details.
Stop a post-process search
Stopping a post-process search causes the base search for that post-process search to stop anywhere it is used. If the same base search is used in multiple panels, it stops in those panels too.
Troubleshoot referenced real-time searches in search head clusters
In a search head clustering (SHC) deployment, if you are referencing a real-time saved search in a dashboard on a search head, the real-time search might not continue to stream data after initial results are returned.
There are two workarounds for this issue.
Option | Example dashboard source code | Performance considerations |
---|---|---|
Use an inline real-time search in the dashboard panel instead. |
<search> <query>index=_internal | stats count </query> <earliest>rt-5m</earliest> <latest>rtnow</latest> </search> |
This type of search runs only when users view the dashboard. However, a new real-time search spawns for each user that accesses the dashboard from the search head or another member. |
Create a scheduled saved search. Use the loadjob command in an inline panel search to update the dashboard with the saved search results.
|
<search> <query> | loadjob savedsearch="admin:search:SavedSearch" </query> </search> |
Only one instance of the saved search runs at the scheduled time regardless of the number of users accessing the dashboard. |
Limitations to search based dashboards
Dashboards with many searches and exceeding 7,000 characters in the Uniform Resource Identifier (URI) do not consistently load and might remain in a "waiting on data" state. A large URI might exceed the limit of various server software used with the Splunk platform, such as NGINX.
Additional search resources
If you are new to the Splunk platform and the search processing language (SPL), start with the Search Tutorial. This tutorial introduces you to the Search and Reporting application. The tutorial guides you through adding data to your Splunk deployment, searching your data, and building simple reports and dashboards.
The Search Manual includes detailed information about creating and optimizing searches, retrieving events, specifying time ranges, and using subsearches.
The Search Reference is a reference guide for the Search Processing Language (SPL). The Search Reference contains a catalog of the search commands with syntax, descriptions, and examples.
Editing Simple XML | Dashboards and forms |
This documentation applies to the following versions of Splunk® Enterprise: 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.1.7, 9.1.8, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.2.4, 9.2.5, 9.3.0, 9.3.1, 9.3.2, 9.3.3, 9.4.0, 9.4.1
Feedback submitted, thanks!