Chain searches together with a base search and chain searches

When you use a separate search for each visualization on a large dashboard, you can use a lot of computing power. When these searches begin with the same initial SPL search sections, you can use these sections as a base search and extend it using additional, chained data source searches that will drive the same visualizations, using less computing power because the base search is only run one time for all of the visualizations. This can be done directly in the visual editor.

For example, three data sources have the following three searches which begin with the same first two search sections:
Search 1

index=_internal | stats count by status | where status < 300 | stats sum(count) as "Success"

Search 2

index=_internal | stats count by status | where status >= 400 | stats sum(count) as "Failed"

Search 3

index=_internal | stats count by status | where status >= 400 | where status < 500 | stats sum(count) as "UserError"

You can create two data sources of type ds.chain that tie them to an existing base search. For example:

Chain search 1

| where status < 300 | stats sum(count) as "Success"

Chain search 2

| where status >= 400

You can further extend a chain search by one additional search. For example:

Chain search 2a

| stats sum(count) as "Failed"

Chain search 2b

| where status < 500 | stats sum(count) as "UserError"

Create a base search

1. Select your visualization. Selected visualizations highlight with a blue outline.
2. Navigate to the Configuration panel.
3. Expand the Data sources section of the Configuration panel.
4. For your base search, select select + Set up annotation data source. The Select data source panel will open.
5. (Optional) You can select an existing search in the Search or Saved search sections.
6. If you're creating a new base search, navigate to the Search section.
7. Select + Create search.
8. (Optional) In the Data source name field, give your search an identifiable name.
9. In the SPL query text box, add your SPL query for your base search.
10. Select Apply and close.

Connect a base search to a chain search

1. Select a visualization that you want to chain to your base search. Selected visualizations highlight with a blue outline.
2. Navigate to the Configuration panel.
3. Expand the Data sources section of the Configuration panel.
4. Select + Set up annotation data source. The Select data source panel will open.
5. Navigate to the Chain search section.
6. Select + Create chain search.
7. (Optional) In the Data source name field, give your search an identifiable name.
8. In the Parent search drop-down list, select your base search. The base search's SPL query will populate in the first search text box.
9. Navigate to the SPL query text box with your chain search's data source name. Add your chain search's SPL query to the text box.
10. Select Apply and close.

You can use this method to create up to 10 searches off of the base search, but you can only create 1 additional chain search by extending an existing chain search. To extend a chain search, choose the first chain search as the parent search instead of selecting your base search.

Adding comments in a chain search

Comments in chain searches have a particular behavior when handling pipes. By default, the Splunk Platform adds a pipe to the beginning of each chain search unless a user initially includes the pipe. When adding a comment, remove the pipe from the start of the chain search or include the comment after the first pipe of the chain search.

Best practices for using comments with chain searches:

The following is an example of not including a pipe at the start of the chain search:

```comment``` chart count

This is an example of how to include the comment after the first pipe of the chain search:

| ```comment``` chart count

Example

There are many different combinations you can use once you've established the base search, for example:

The base search becomes:

index=_internal | stats count by status

Search 1 is now the following: base search + Chain search 1

Search 2 is now the following: base search + Chain search 2 + Chain search 2a

Search 3 is now the following: base search + Chain search 2 + Chain search 2b

You can extend many independent chain searches from the base search, and you can extend many second level chains that use the first level chain as their primary data source, but you cannot have a third level of chains that use the a second level chain as a primary data source.

This functionality is similar to the way that you might have used post-process searches using Simple XML.

You can use tokens in both base and chain searches. Any token you create can be used in a search of type ds.chain, but time-related tokens can only be used in the base search.

You cannot specify any properties such as queryParameters, refresh , and refreshType for chain searches. These are inherited from those set in the base search, or from settings in the defaults section. When a base search is refreshed, or its SPL search is changed, the associated chain searches will also be refreshed.

An example of the anatomy of the chain search

In the following example, there are four searches. One base search and three chain searches. One chain search relies on the base search, and the other two rely on the first chain search.

ds.chain example

Expand this window to see the dashboard definition of the example.

{
    "title": "Base and Post Search Example",
    "description": "",
    "defaults": {},
    "visualizations": {
        "viz_2AFU2XSx": {
            "context": {},
            "dataSources": {
                "primary": "ds_g9BqIN8h"
            },
            "options": {
                "legendTruncation": "ellipsisMiddle",
                "showRoundedY2AxisLabels": false,
                "showY2MajorGridLines": true,
                "y2AxisAbbreviation": "off",
                "yAxisAbbreviation": "off"
            },
            "title": "Base Data",
            "type": "splunk.column"
        },
        "viz_61GNJM6Y": {
            "context": {},
            "dataSources": {
                "primary": "ds_Ah56LDFf"
            },
            "options": {
                "showSparklineAreaGraph": false,
                "sparklineValues": "> primary | seriesByTypes(\"number\", \"string\")"
            },
            "title": "Average",
            "type": "splunk.singlevalue"
        },
        "viz_9zvXENZu": {
            "context": {
                "formattedConfig": {
                    "number": {
                        "precision": 0,
                        "thousandSeparated": false,
                        "unitPosition": "after"
                    },
                    "string": {
                        "unitPosition": "after"
                    }
                }
            },
            "dataSources": {
                "primary": "ds_OSNBwGNz"
            },
            "options": {
                "count": 20,
                "tableFormat": {
                    "data": "> table | formatByType(formattedConfig)"
                }
            },
            "title": "Chained Data",
            "type": "splunk.table"
        },
        "viz_UZfgjCjy": {
            "options": {},
            "type": "abslayout.line"
        },
        "viz_aGYxaiBz": {
            "options": {},
            "type": "abslayout.line"
        },
        "viz_asyqhgft": {
            "options": {},
            "type": "abslayout.line"
        },
        "viz_gnZcFdBS": {
            "context": {},
            "dataSources": {
                "primary": "ds_dFHviA1E"
            },
            "options": {
                "showSparklineAreaGraph": false,
                "sparklineValues": "> primary | seriesByTypes(\"number\", \"string\")"
            },
            "title": "Count",
            "type": "splunk.singlevalue"
        }
    },
    "dataSources": {
        "ds_Ah56LDFf": {
            "name": "Chain search 2 extends chain search 1",
            "options": {
                "extend": "ds_OSNBwGNz",
                "query": "| stats avg"
            },
            "type": "ds.chain"
        },
        "ds_OSNBwGNz": {
            "name": "Chain search 1 extends base search",
            "options": {
                "extend": "ds_g9BqIN8h",
                "query": "| where count > 10"
            },
            "type": "ds.chain"
        },
        "ds_dFHviA1E": {
            "name": "Chain search 3 extends chain search 2",
            "options": {
                "extend": "ds_OSNBwGNz",
                "query": "|  stats count"
            },
            "type": "ds.chain"
        },
        "ds_g9BqIN8h": {
            "name": "Base search",
            "options": {
                "query": "index=_internal \n|  top 100 sourcetype"
            },
            "type": "ds.search"
        }
    },
    "layout": {
        "layoutDefinitions": {
            "layout_1": {
                "structure": [
                    {
                        "item": "viz_2AFU2XSx",
                        "position": {
                            "h": 290,
                            "w": 350,
                            "x": 20,
                            "y": 30
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_9zvXENZu",
                        "position": {
                            "h": 310,
                            "w": 440,
                            "x": 420,
                            "y": 20
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_61GNJM6Y",
                        "position": {
                            "h": 150,
                            "w": 150,
                            "x": 940,
                            "y": 20
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_gnZcFdBS",
                        "position": {
                            "h": 150,
                            "w": 150,
                            "x": 940,
                            "y": 200
                        },
                        "type": "block"
                    },
                    {
                        "item": "viz_UZfgjCjy",
                        "position": {
                            "from": {
                                "item": "viz_2AFU2XSx",
                                "port": "e"
                            },
                            "to": {
                                "item": "viz_9zvXENZu",
                                "port": "w"
                            }
                        },
                        "type": "line"
                    },
                    {
                        "item": "viz_aGYxaiBz",
                        "position": {
                            "from": {
                                "item": "viz_9zvXENZu",
                                "port": "e"
                            },
                            "to": {
                                "item": "viz_61GNJM6Y",
                                "port": "w"
                            }
                        },
                        "type": "line"
                    },
                    {
                        "item": "viz_asyqhgft",
                        "position": {
                            "from": {
                                "item": "viz_9zvXENZu",
                                "port": "e"
                            },
                            "to": {
                                "item": "viz_gnZcFdBS",
                                "port": "w"
                            }
                        },
                        "type": "line"
                    }
                ],
                "type": "absolute"
            }
        },
        "tabs": {
            "items": [
                {
                    "label": "New tab",
                    "layoutId": "layout_1"
                }
            ]
        }
    }
}

If you are using lookups in your search, make sure the lookup permissions are set to "global".

Best practices for creating chain searches

Use these best practices to make sure that chain searches work as expected.

Use a transforming base search

A base search should be a transforming search that returns results formatted as a statistics table. For example, searches using the following commands are transforming searches: stats, chart, timechart, and geostats, among others. For more information on transforming commands, see About transforming commands in the Search Manual.

Fixing search result and timeout issues

Non-transforming base searches can cause search result and timeout issues. If you observe the following issues in a dashboard, check the base search to make sure that it is a transforming search:

• no results returned
• event retention
• client timeout
• the collect command

No results returned
If the base search is a non-transforming search, you must explicitly state in the base search what fields will be used in the chain search using the | fields command. For example, if your chain search will search for the top selling buttercup game categories over time, you would use a search command similar to the following.

| fields _time, categoryId, action

Event retention
If the base search is a non-transforming search, the Splunk platform retains only the first 500,000 events that it returns. A chain search does not process events in excess of this 500,000 event limit, silently ignoring them. This can generate incomplete data for the chain search.

This search result retention limit matches the max_count setting in limits.conf. The setting default is 500,000.

Client timeout
If the chain search takes too long, it can exceed the Splunk Web client timeout value of 30 seconds.

The collect command

The collect command does not work with chain searches when used in the base search.

For more information about transforming searches, see transforming commands and searches in the Search Manual.

Do not reference fields in chain searches that are not referenced in the base search

A chain search depends entirely on the fields present in the base search. If you are not referencing a particular field in the base search, do not reference it in the chain search. Fields used in transforming commands will automatically be available for chain searches. When transforming commands are not used in a base search, fields without a reference in the base search appear null in a chain search. The chain search returns no results in this case.

Utilize fewer base searches

Using fewer base searches can improve your dashboard's performance. For example, one base search is often more efficient than multiple base searches.

Limit base search results and chain complexity

Passing a large number of search results to a chain search can cause server timeout issues. In this scenario, consider adjusting the base search to reduce the number of results and fields that it returns. You can also consider reducing the complexity of chain searches on the base search results.

You can use a single chain search from a base search to generate results or you can generate multiple chain searches together.

Base and chain refresh behavior

Specific changes occurring in a base or chain will impact the refresh behavior of the base and chain. The following describes the actions that initiate a refresh on a base, chain, or both.

Description	Refresh behavior
Token value changes in the base.	Entire base and chain tree refreshes.
Token value changes in only the chain. No change occurs in the base.	Only the chain search refreshes. The base does not refresh.
Auto refresh interval is set on a base or chain.	Entire base and chain tree refreshes.
Manual refresh is triggered on a base or chain.	Entire base and chain tree refreshes.