Splunk® Dashboards App

Splunk Dashboards app (beta) for Enterprise and Cloud

Download manual as PDF

Download topic as PDF

Add and configure data sources

In your dashboard definition, each data source stanza will have a data source type associated with it. For example, when you create the search in the Configuration panel, the resulting data source is of type ds.search. Each type follows a similar format, but operates in a different way. To see the configuration options available for each data source type, see Data source options.


ds.search

The data source ds.search is used for ad hoc searches that you create in the visual editor. When you create a data source using a search in the visual editor, the ds.search data source is assigned a unique ID, followed by the data source type, and any options you would like to make available.

The only option that must be set is the query option, which is an SPL search.


{
"dataSources": {
       "<search_name>": { 
              "type": "ds.search",
              "options": {
                      "query": "<your_SPL_search_>"
             }
        }
   }
}

The data source, BCsearch uses the search,

index=_internal | head 500

in the query field in the following example to return results that can be displayed by adding the ID of the dataSource, to a visualization module.

{
"dataSources": {
	"BCsearch": {
		"type": "ds.search",
		"options": {
			"query": "index=_internal | head 500",
			"queryParameters": {
				"earliest": "0",
				"latest": "now"
			}
		}
	}
   }
}

In order to find it more easily in the dashboard definition, you can give your data source stanza a title, separate from the unique ID, you can use the "name" field. You can also rename the ID, as long as it is only used one time in that dashboard definition.

When setting options in the options field, boolean values and numbers do not require quotes. Strings, such as the hex color codes, do require quotes. Commas must come after each entry except the last. The list of options you use must be enclosed in curly braces followed by a comma.

In the following example of a dataSources section, there is one data source stanza with the unique ID of "search_1".

	"dataSources": {
		"search_1": {
			"type": "ds.search",
			"options": {
				"queryParameters": {
					"earliest": "$TimeRange.earliest$",
					"latest": "$TimeRange.latest$"
				},
				"query": "index=_internal | top limit=100 sourcetype | eval percent = round(percent,2)"
			},
			"refreshType": "delay",
			"refresh": "10s"
		}
	}

Use a ds.chain with a base search to chain searches together

You will often find that you use the same basic search when creating data source searches. Instead of repeating the first pipe query, for example index=_internal sourcetype=* for every data source, you can chain searches off of this search. When you do this, you're using the original search and creating up to two chain searches that will use the original search. These chain searches each have separate data sources which will demand fewer computing resources than relying on a single search for many visualizations. The data source with the original search is called a base search, and the data source type is either ds.search or ds.savedSearch. The additional data sources are called chain searches, and their data source type is ds.chain. These searches use the extend field to tie them to either the base search by its ID, or one other chain search by its ID. This functionality is similar to the way that you might have used post-process searches using Simple XML.

You can extend as many independent chain searches from the base search as you want, and you can have as many second level chains that use the first level chain as the primary data source, but you cannot have a third level of chains that use the a second level chain as a primary data source.

You can't use tokens in data sources of type ds.chain.

ds.chain data sources can only be created in the source editor, and you cannot specify any options such as queryParameters, refresh , and refreshType. These are inherited from the base search.

An example of the anatomy of the chain search

In the following example, there are now four searches. One base search and three chain searches. One chain search relies on the base search, and the other two rely on the first chain search.

Three single value visualizations using a base and two chain dataSources

ds.chain example

Expand this window to see the dashboard definition of the example.

     {
    "layout": {
        "type": "absolute",
        "options": {},
        "structure": [
            {
                "item": "viz_2AFU2XSx",
                "type": "block",
                "position": {
                    "x": 20,
                    "y": 30,
                    "w": 350,
                    "h": 290
                }
            },
            {
                "item": "viz_9zvXENZu",
                "type": "block",
                "position": {
                    "x": 420,
                    "y": 20,
                    "w": 440,
                    "h": 310
                }
            },
            {
                "item": "viz_61GNJM6Y",
                "type": "block",
                "position": {
                    "x": 940,
                    "y": 20,
                    "w": 150,
                    "h": 150
                }
            },
            {
                "item": "viz_gnZcFdBS",
                "type": "block",
                "position": {
                    "x": 940,
                    "y": 200,
                    "w": 150,
                    "h": 150
                }
            },
            {
                "item": "viz_UZfgjCjy",
                "type": "line",
                "position": {
                    "from": {
                        "item": "viz_2AFU2XSx",
                        "port": "e"
                    },
                    "to": {
                        "item": "viz_9zvXENZu",
                        "port": "w"
                    }
                }
            },
            {
                "item": "viz_aGYxaiBz",
                "type": "line",
                "position": {
                    "from": {
                        "item": "viz_9zvXENZu",
                        "port": "e"
                    },
                    "to": {
                        "item": "viz_61GNJM6Y",
                        "port": "w"
                    }
                }
            },
            {
                "item": "viz_asyqhgft",
                "type": "line",
                "position": {
                    "from": {
                        "item": "viz_9zvXENZu",
                        "port": "e"
                    },
                    "to": {
                        "item": "viz_gnZcFdBS",
                        "port": "w"
                    }
                }
            }
        ]
    },
    "dataSources": {
        "ds_g9BqIN8h": {
            "type": "ds.search",
            "options": {
                "query": "index=_internal \n|  top 100 sourcetype"
            },
            "name": "Base search"
        },
        "ds_OSNBwGNz": {
            "type": "ds.chain",
            "options": {
                "query": "| where count > 10",
                "extend": "ds_g9BqIN8h"
            },
            "name": "Chain search 1 extends base search"
        },
        "ds_Ah56LDFf": {
            "type": "ds.chain",
            "options": {
                "query": "| stats avg",
                "extend": "ds_OSNBwGNz"
            },
            "name": "Chain search 2 extends chain search 1"
        },
        "ds_dFHviA1E": {
            "type": "ds.chain",
            "options": {
                "query": "|  stats count",
                "extend": "ds_OSNBwGNz"
            },
            "name": "Chain search 3 extends chain search 2"
        }
    },
    "description": "",
    "title": "Base and Post Search Example",
    "visualizations": {
        "viz_2AFU2XSx": {
            "type": "viz.column",
            "options": {},
            "dataSources": {
                "primary": "ds_g9BqIN8h"
            },
            "title": "Base Data"
        },
        "viz_9zvXENZu": {
            "type": "viz.table",
            "options": {},
            "dataSources": {
                "primary": "ds_OSNBwGNz"
            },
            "title": "Chained Data"
        },
        "viz_61GNJM6Y": {
            "type": "viz.singlevalue",
            "options": {},
            "title": "Average",
            "dataSources": {
                "primary": "ds_Ah56LDFf"
            }
        },
        "viz_gnZcFdBS": {
            "type": "viz.singlevalue",
            "options": {},
            "title": "Count",
            "dataSources": {
                "primary": "ds_dFHviA1E"
            }
        },
        "viz_UZfgjCjy": {
            "type": "abslayout.line",
            "options": {}
        },
        "viz_aGYxaiBz": {
            "type": "abslayout.line",
            "options": {}
        },
        "viz_asyqhgft": {
            "type": "abslayout.line",
            "options": {}
        }
    }
}

Use a transforming base search

A base search should be a transforming search that returns results formatted as a statistics table. For example, searches using the following commands are transforming searches: stats, chart, timechart, and geostats, among others. For more information on transforming commands, see About transforming commands in the Search Manual.

What not to do when building a chain search

To get the most out of your chain searches, it's best to avoid the following practices:

  • Using the collect command in a base search
  • The collect command does not work with chain searches when used in the base search.

  • Reference fields in chain searches that are not referenced in the base search.
    A chain search depends entirely on the fields present in the base search. If you are not referencing a particular field in the base search, do not reference it in the chain search. Fields used in transforming commands will automatically be available for chain searches. When transforming commands are not used in a base search, fields without a reference in the base search appear null in a post-process search. The post-process search returns no results in this case.
  • Use unnecessarily complex chain searches
    Passing a large number of search results to a chain search can cause server timeout issues. In this scenario, consider adjusting the base search to reduce the number of results and fields that it returns. You can also consider reducing the complexity of chain operations on the base search results. You can use a single chain search from a base search to generate results or you can generate multiple chain searches together. Inputs and tokens are not supported in chain searches or in data sources used as base searches.
  • Using a non-transforming base search
    Non-transforming base searches can cause search result and timeout issues. If you observe the following issues in a dashboard, check the base search to make sure that it is a transforming search:
    • No results returned
      If the base search is a non-transforming search, you must explicitly state in the base search what fields will be used in the chain search using the

      | fields

      command. For example, if your chain search will search for the top selling buttercup game categories over time, you would use a search command similar to the following:

      | fields _time, categoryId, action

    • Event retention
      If the base search is a non-transforming search, the Splunk platform retains only the first 500,000 events that it returns. A chain search does not process events in excess of this 500,000 event limit, silently ignoring them. This can generate incomplete data for the chain search. This search result retention limit matches the max_count setting in limits.conf. The setting defaults to 500,000.
    • Client timeout
      If the post-processing operation takes too long, it can exceed the Splunk Web client timeout value of 30 seconds.

Use ds.savedSearch to use reports and saved searches

Use the ds.savedSearch to bring in reports, or saved searches, from within the Splunk Dashboards app or from other apps. You can use the saved search data source to schedule these searches to run on a particular frequency and store the results, which lightens processing loads and concurrent search limits. For example, if fifty users access a particular dashboard, panels backed by scheduled saved searches will not cause the searches to run fifty times, while panels backed by searches of the the type ds.search might.

If you have dashboards in the Search & Reporting app that use reports, they will transfer as ds.savedSearch when you open that dashboards the Splunk Dashboards app.

Imported and native reports

You must reference a saved search or report by its name by using the ref property. To specify which app the saved search or report belongs to, use the app property. If no app is specified, It's assumed the report came from the Search & Reporting app.

Access saved searches that live in the Search & Reporting app

The following procedure shows you how to access a saved search that lives in the Search & Reporting app.

  1. Export your dashboard from the Search & Reporting app, or create a new one in the Splunk Dashboards app.

  2. In the Search & Reporting app, find the saved search you want to use.
    You can find it in Settings > Searches, reports, and alerts. In this section, saved searches are called reports. Make a note of the exact names of the reports. The exact name must be set in the ref option of the data source stanza. You may want to use a text editor to keep track of the names.

  3. Add a ds.savedSearch stanza to the dashboard definition in the Splunk Dashboards app.
    You may want to change the unique ID to easily find it in the dashboard definition. Note the data source type is ds.savedSearch For example:
    "reportNoScheduleNoRefresh": {
    	"type": "ds.savedSearch",
    	"options": {
    		"ref": "Top 100 sourcetypes in the last 24 hours"
    			}
    		},
    

The default assumption is that the saved search you're referencing lives in the Search & Reporting app. If you created your saved search within the Splunk Dashboards app, or in any app other than Search & Reporting, you must use the app option and set it to the app where the saved search was saved. For example, if you created a saved search in the Splunk Dashboard app, the stanza would look like the following:

"reportNoScheduleNoRefresh": {
	"type": "ds.savedSearch",
	"options": {
		"ref": "Top 100 sourcetypes in the last 24 hours"
                "app": "splunk-dashboard-app"
			}
		},

You can look to the URL while viewing the report to determine what app it is saved in. You can also see information about your searches in the visual editor in the Configuration panel in the Data Configurations section.

The Data Configurations section of the Configuration panel

When you click on the report, you will see the following:

  • The saved search name in the Data Source Name field.
  • The query used in the Saved Search with SPL window.
  • The time range set in the stanza.
  • The creator of the saved search.
  • The location of the original saved search.
  • The permission status.
  • The last time the saved search was edited.

The window that opens when you click on the report in the Data Configurations section of the Configuration panel.

You can't change any of the information using this panel, however, you can click Open in Reports to open the original search in the app where the saved search exists.

ds.savedSearch options

Unlike ds.search, the ds.savedSearch data source type does not respect the name or query options. The only options that you can use are ref and app, as well as refresh and refreshType for searches that are not scheduled searches. For example, the following saved search refreshes at 5 second intervals:

"reportNoScheduleWithRefresh": {
		"type": "ds.savedSearch",
		"options": {
			"ref": "Current Time",
		        "refresh": "5s",
			"refreshType": "interval"
		},

Scheduled saved searches

If a you are referencing a saved search that is scheduled, that schedule will be respected regardless of whether you set the refresh or refreshType options. For example, if a report from the Search & Reporting app has been scheduled to run at the top of every minute, setting the refresh interval to 5 seconds in the JSON stanza will have no effect, and the scheduled search will continue to run at the top of every minute. The same is true for any settings that apply to the data source type in the defaults section of the dashboard definition.

If the report is a scheduled search, but its first scheduled run has not yet completed, the search will run automatically the first time it's added to the dashboard definition.

Use a saved search as a base search for scheduled chain searches

You can use the ds.savedSearch data source as a base search in the same way that you can use ds.search as a base search. The ds.chain data source accepts all of the options that it would if the base search was of type ds.search. For example:

		"baseReportNoScheduleWithRefresh": {
			"type": "ds.savedSearch",
			"options": {
				"ref": "Current Time",
				"refresh": "5s",
				"refreshType": "interval"
			}
		},
		"postReportNoScheduleWithRefresh": {
			"type": "ds.chain",
			"options": {
				"query": "| eval count=count-1500000000",
				"extend": "baseReportNoScheduleWithRefresh"
			},
			"name": "Data Source Name: Post Report - No Schedule, With Refresh"
		},

For more information on chain searches, see ds.chain

Remember, if you have the cancelJobsOnFocusLoss option, or any other options set in the defaults section of the dashboard definition, it most likely affects the data source of type ds.search. Since ds.savedSearch is of a different type, these searches will not be affected by that setting and will continue to run in the background. It is important that you do not set cancelJobsOnFocusLoss for the ds.savedSearch data type since these searches are usually scheduled and depend on being run uninterrupted.

ds.test

The ds.test data source is used for sample data and is structured by first providing a unique name for the data source, followed by the data source type, options,data, column values, and fieldst . For more information on dataSource options see dataSource options.

ds.test data sources can only be created in the source editor

You create column events and field values manually. The order of the column and field options is unimportant, but you must have the same number of columns as values. ds.test is structured in the following way:

ds.test example

The following is an actual example of how ds.test can be used.

 
   "dataSources":{ 
      "test_search":{ 
         "type":"ds.test",
         "options":{ 
            "data":{ 
               "columns":[ 
                  [ 
                     "splunkd",
                     "splunkd_ui_access",
                     "splunkd_access",
                     "splunk_web_access",
                     "scheduler",
                     "splunk_web_service"
                  ],
                  [ 
                     "600",
                     "525",
                     "295",
                     "213",
                     "122",
                     "19"
                  ],
                  [ 
                     "87.966380",
                     "50.381304",
                     "60.023780",
                     "121.183272",
                     "70.250513",
                     "90.194752"
                  ]
               ],
               "fields":[ 
                  { 
                     "name":"sourcetype"
                  },
                  { 
                     "name":"count",
                     "type_special":"count"
                  },
                  { 
                     "name":"percent",
                     "type_special":"percent"
                  }
               ]
            },
            "meta":{ 
            }
         }
      }
   }


Data source options

The options field of a data source stanza is where you can set various properties. Following the JSON format, each property setting must be enclosed in quotes unless it is a boolean value (true, false) or a number. Options other than the last one set must end in a comma.

The following table lists the options that are available for you to use to modify your dataSource stanzas:

option type default description
app string search Only for ds.savedSearch. Define the app that is associated with a report or saved search that you want to use.
query string N/A Use the query option to write your ad-hoc, base, or chain search.
queryParameters.earliest string N/A Specify the earliest time to search for events. Choose from year (y), month (m), week (w), day (d), minute, (m), or second (s), or 0 for all time. For example, if you want to run a search that runs for all time, see The queryParameters example.
queryParameters.latest string N/A Specify the latest time to search for events. Choose from year (y), month (m), week (w), day (d), minute, (m), or second (s), or 0 for all time. For example, if you want to run a search that runs for all time, see The queryParameters example.
queryParameters.timezone string The default is the timezone the user is in. Set the timezone for the search parameters to run in.
ref string N/Z Only for ds.savedSearch. Enter the exact name of the report you are using. This will allow the Dashboard app to pull the report from S&R.
refresh string N/A Specify the refresh interval with a time expression. For example, "5s" for five seconds or "1m" for one minute. See refreshType and refresh example.
refreshType (delay | interval) delay Indicate the starting time for a search to refresh. Use "delay" to start the countdown to refresh when the search is done. Use "interval" to count down when the search is dispatched. See refreshType and refresh example.

The queryParameters option example

The following example allows a user to specify a time span for a search to run, and also changes the title of the resulting pie chart. It uses the option queryParameters:

source code

Expand this box to see the complete dashboard definition. You can copy/paste it into your own instance to view how this inputs work with queryParameters.

{
	"visualizations": {
		"chart_1": {
			"type": "viz.pie",
			"options": {},
			"dataSources": {
				"primary": "search_1"
			},
			"description": "Chart of Top Sourcetypes between $TimeRange.earliest$ and $TimeRange.latest$"
		}
	},
	"dataSources": {
		"search_1": {
			"type": "ds.search",
			"options": {
				"queryParameters": {
					"earliest": "$TimeRange.earliest$",
					"latest": "$TimeRange.latest$"
				},
				"query": "index=_internal | top limit=100 sourcetype | eval percent = round(percent,2)"
			}
		}
	},
	"inputs": {
		"input_1": {
			"type": "input.timerange",
			"title": "Select a time:",
			"options": {
				"token": "TimeRange",
				"defaultValue": "-24h,now"
			}
		}
	},
	"layout": {
		"options": {
			"submitButton": false,
			"height": 1250,
			"display": "auto-scale",
			"width": 1200
		},
		"globalInputs": [
			"input_1"
		],
		"structure": [
			{
				"item": "chart_1",
				"type": "block",
				"position": {
					"x": 250,
					"y": 80,
					"w": 680,
					"h": 530
				}
			}
		],
		"type": "absolute"
	},
	"title": "Time Picker Input Example",
	"description": "Add a time range picker to modify a search time span."
}

refreshType and refresh options example

If you add the refreshType and refresh options to your data source, the visualization will refresh automatically at the interval you specify. In this case, refreshType is set to delay the refresh until the end of the search at an interval of 10 seconds.

Specifying these settings in the stanza overrides any refresh and refreshType setting in the defaults section of the dashboard definition.

	"dataSources": {
		"search_1": {
			"type": "ds.search",
			"options": {
				"queryParameters": {
					"earliest": "$TimeRange.earliest$",
					"latest": "$TimeRange.latest$"
				},
				"query": "index=_internal | top limit=100 sourcetype | eval percent = round(percent,2)"
			},
			"refreshType": "delay",
			"refresh": "10s"
		}
	},

The following dashboard definition is a modified version of the queryParameters dashboard. The refresh and refreshType options have been added.

queryParameters and refresh options example

source code

Expand this box to see the complete dashboard definition. You can copy/paste it into your own instance.

{
	"visualizations": {
		"chart_1": {
			"type": "viz.pie",
			"options": {},
			"dataSources": {
				"primary": "search_1"
			},
			"description": "Chart of Top Sourcetypes between $TimeRange.earliest$ and $TimeRange.latest$"
		}
	},
		"dataSources": {
		"search_1": {
			"type": "ds.search",
			"options": {
				"queryParameters": {
					"earliest": "$TimeRange.earliest$",
					"latest": "$TimeRange.latest$"
				},
				"query": "index=_internal | top limit=100 sourcetype | eval percent = round(percent,2)"
			},
			"refreshType": "delay",
			"refresh": "10s"
		}
	},
	"inputs": {
		"input_1": {
			"type": "input.timerange",
			"title": "Select a time:",
			"options": {
				"token": "TimeRange",
				"defaultValue": "-24h,now"
			}
		}
	},
	"layout": {
		"options": {
			"submitButton": false,
			"height": 1250,
			"display": "auto-scale",
			"width": 1200
		},
		"globalInputs": [
			"input_1"
		],
		"structure": [
			{
				"item": "chart_1",
				"type": "block",
				"position": {
					"x": 250,
					"y": 80,
					"w": 680,
					"h": 530
				}
			}
		],
		"type": "absolute"
	},
	"title": "Time Picker Input Example",
	"description": "Add a time range picker to modify a search time span."
}
Last modified on 21 September, 2020
PREVIOUS
How the dashboard definition is structured in the source editor
  NEXT
Use layout options to modify your dashboard canvas with the source editor

This documentation applies to the following versions of Splunk® Dashboards App: 0.6.0, 0.7.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters