Migrate from hybrid search to federated search
Do you run hybrid searches from your Splunk Enterprise search head that combine data from your Splunk Enterprise instance with data from a Splunk Cloud Platform environment? If so, you should migrate your hybrid searches to the federated search solution. Federated search has several advantages over hybrid search. A move to federated search will expand what you can do with cross-deployment searches.
Comparing hybrid search and federated search
The following table shows you how hybrid search and federated search match up.
Feature | Hybrid Search | Federated Search |
---|---|---|
Environments spanned in a search | Searches can span a single Splunk Enterprise environment and a single Splunk Cloud Platform environment. | Searches can span multiple Splunk Enterprise environments and multiple Splunk Cloud Platform environments. |
Ad-hoc search | Supported | Supported |
Scheduled search | Not supported | Supported |
Workload management (WLM) | Not supported | Supported |
SPL support | No special syntax required. All commands supported. | Requires special syntax. Does not support Generating commands other than search and from .
|
Security (RBAC) | All security is enforced at the Splunk Enterprise search head | Security is enforced on the local and remote search heads involved in a federated search. On the remote search head, you can apply access control at the service account for the federated provider. On the local federated search head, you can apply additional, more granular access controls at the remote dataset level by setting federated index restrictions. |
Search Head architecture | For hybrid search, the Splunk Cloud Platform requires a single search head. Hybrid search does not support Splunk Cloud Platform environments with search head cluster configurations. | Federated search supports all search management tier architecture options and combinations. |
Version compatibility and upgrades | There are strict version dependencies for hybrid search between Splunk Enterprise and Splunk Cloud Platform environments. An upgrade on either side can break hybrid searches until the corresponding deployment is upgraded to a compatible version. | You need to have Splunk Enterprise 8.2 or later, and Splunk Cloud Platform 8.2.2104 or later. There isn't a strict versioning dependency between the two platforms. Splunk Cloud Platform upgrades won't break federated searches. |
Operability | To enable and configure hybrid search between a Splunk Enterprise environment and Splunk Cloud Platform environment, you must contact your Splunk representative. | In most cases, setup of federated search is possible entirely through self-service means. |
Move to federated search
The move to federated search requires that you follow a few self-service steps. Afterwards, you can run federated searches that combine data from your local Splunk Enterprise deployment and a Splunk Cloud Platform environment.
- Designate the Splunk Cloud Platform environment as a federated provider.
- Define a separate federated index for each remote dataset you want to search on the federated provider. Currently you can designate indexes and saved searches as remote datasets.
- Convert your hybrid searches into federated searches.
See About federated search for an overview of federated search and terminology definitions.
Create a federated provider definition for your Splunk Cloud Platform environment
You create a federated provider definition for your Splunk Cloud Platform environment through the Federated Provider page. The Federated Provider page is available in Settings. These settings determine how the federated search head on your Splunk Enterprise deployment collaborates with the remote search heads on your federated provider to run a federated search.
See Define a federated provider.
When you set up a Splunk Cloud Platform environment as a federated provider, you:
- Help your local federated search head connect to the remote federated provider
- Supply the Splunk Cloud Platform environment host name and port number.
- Provide a service account user id and password
- This dedicated user account is set up on the federated provider. It allows the federated search head to search datasets on the federated provider. You can limit what indexes and knowledge objects this user account can access by adjusting its role-based security.
- Determine whether you are using local or remote knowledge objects
- A pair of settings let you determine whether the knowledge objects used in your federated searches with a given provider come from the local federated search head on your Splunk Enterprise deployment or the remote search head on the federated provider.
Next, you set up federated index definitions for the datasets that you want to search on the Splunk Cloud Platform environment.
Create federated indexes and map them to remote datasets
Each federated index you create maps to a specific remote dataset on a federated provider. When you write a federated search, you use federated index references to search those remote datasets. Currently you can designate indexes and saved searches as remote datasets. See Create a federated index.
When you create a federated index definition, you:
- Name a federated provider
- Supply the name of the federated provider that contains the dataset you want the federated index to map to.
- Identify a dataset
- Name the dataset to which you want the federated index to map. Currently, indexes and saved searches can be set up as datasets for federated searches.
If you are not sure which datasets to use for your federated index definitions, look at the hybrid searches you commonly run. If those hybrid searches reference remote indexes on the federated provider, use those indexes in your federated index definitions.
You can manage the access that your users have to the remote datasets to which your federated indexes map. Set up role-based restrictions to the federated indexes, just as you would for any other index. See Create and manage roles with Splunk Web in Securing the Splunk Platform.
Write and run federated searches
After you set up your Splunk Cloud Platform environment as a federated provider and define federated indexes that are mapped to datasets on that federated provider, you are ready to write and run federated searches. If you have existing hybrid searches that you want to convert into federated searches, you need to convert your remote index references into federated search references.
To reference a federated index that maps to a remote index dataset, use this syntax:
search index=federated:<federated_index_for_remote_index>
You can also reference saved search datasets, in your federated searches, if you have created federated indexes that map to them. To reference a federated index that maps to a remote saved search dataset, use this syntax:
| from federated:<federated_index_for_remote_saved_search>
See Run federated searches for more information about writing federated searches and about restrictions on federated searches.
About federated search | Define a federated provider |
This documentation applies to the following versions of Splunk® Enterprise: 8.2.0, 8.2.1
Feedback submitted, thanks!