About federated search
You can run federated searches to search datasets outside of your local Splunk deployment. From your local search head, federated search gives you a holistic view of datasets across multiple Splunk platform deployments.
Federated search is topology-agnostic, so it works despite the complexity of the Splunk platform deployments involved. You can run a federated search across any remote Splunk Cloud Platform or Splunk Enterprise deployment, whether it has a single search head or a search head cluster.
Federated search is currently unavailable for regulated (FedRAMP, PCI, and HIPAA) Splunk Cloud Platform environments. Federated search is also unavailable for Splunk Cloud Platform environments in Google Cloud regions.
Components of a typical federated search setup
Federated search introduces a set of terms. Familiarize yourself with them before you attempt to dig into setting up and running federated searches.
A search of one or more remote datasets on one or more federated providers.
The Splunk platform deployment from which you perform federated searches. The federated search head for your federated search resides on your local deployment.
In this context, "local" does not refer to your physical location. If you are in London and are logging into a Splunk platform deployment located in New York City when you run a federated search, that New York City deployment is the local deployment for your federated search.
Federated search head
A search head residing on your local deployment that initiates federated searches. Contains federated index definitions for standard mode federated providers.
A remote Splunk platform deployment. Contains the data that you search with your federated searches.
Before you can run federated searches, you must create federated provider definitions on the local Splunk platform deployment. A federated provider definition serves several purposes:
- It enables the federated search head to make network connections to the federated provider and run searches on a remote search head on that provider through a service account.
- It determines whether the federated provider runs in standard or transparent mode.
- It determines which set of knowledge objects the federated provider applies to federated searches.
Remote search head
A search head on a federated provider.
An index you create on your federated search head to run federated searches over standard mode federated providers. Each federated index maps to a specific remote dataset on a standard mode federated provider. Federated indexes can't be set up to ingest data or events. They provide a logical mapping to remote datasets, but do not contain data. See Create a federated index.
A dataset on a standard mode federated provider. Currently, only events indexes and saved searches qualify as remote datasets.
How federated search works
The federated search process works similarly to distributed search. The initial processing of a search is handled by the indexers of a Splunk platform deployment, and then the results are aggregated on the search head for that deployment to produce a final result set.
Federated searches, however, are broken up into parts that are processed locally and parts that are processed remotely. The remote processing takes place on one or more federated providers.
For example, say you have a simple federated search that involves only one federated provider. In this case, the federated search process sends the remote portion of the search to the federated provider. On the federated provider, the search is processed independently by the remote search head and its indexers. The remote search head then sends the results back to the federated search head on the local Splunk platform deployment, where the local search head aggregates the remote results into the final result set for the complete federated search.
The following diagram illustrates a federated search over a remote Splunk platform deployment. The remote deployment is set up as a standard mode federated provider. The federated provider has an events index dataset that is available for federated searches. On the local Splunk platform deployment, a federated index on the federated search head is mapped to the remote datasets.
If the federated provider in this example were a transparent mode federated provider, it would not be associated with a federated index.
A simple federated search for this setup might look like this:
index=federated:provider1_fedindex1 | stats count
This search references a federated index named
provider1_fedindex1 federated index is mapped to the remote dataset stored on Federated Provider 1. The remote search head uses this mapping to send the events in the remote index to the federated search head on your local deployment. The federated search head runs the
stats count operation on those events. When this
stats count aggregation is complete, the federated search head presents the results without additional processing, as there are no additional datasets involved in the search.
See Run federated searches to learn how to write federated searches.
Supported federated search configurations
The types of federated search configurations you can run depends on the type of Splunk platform deployment you are running locally.
|Type of the local Splunk platform deployment||Splunk platform deployment types you can set up as federated providers|
|Splunk Cloud Platform (version 8.1.2012 or higher)||Splunk Cloud Platform (version 8.1.2012 or higher)|
|Splunk Enterprise (version 8.2.0 or higher)||Splunk Cloud Platform (version 8.2.2104 or higher)|
Splunk Enterprise (version 8.2.0 or higher)
If your local deployment is on Splunk Cloud Platform, you can run federated searches over other Splunk Cloud Platform deployments. If your local deployment is on Splunk Enterprise, you can run federated searches over remote Splunk Cloud Platform deployments and remote Splunk Enterprise deployments.
If you have a Splunk Enterprise deployment that is lower than 8.2 and want to run federated searches without upgrading the entire deployment, you can upgrade a single search head in that deployment to 8.2 and run federated searches from that search head.
Federated search for Splunk Enterprise is limited to Splunk Enterprise deployments that have been built over the Linux operating system. This restriction does not affect Splunk Cloud Platform deployments.
About standard and transparent mode
When you define a federated provider, you must decide what mode you want that provider to use. Federated provider modes offer different federated search experiences, and you must choose the mode that best fits your needs.
You have two federated provider mode options: standard and transparent.
- You might choose standard mode if you want to restrict access to datasets and search dataset types that can be invoked by the
fromcommand, such as saved searches.
- You might choose transparent mode if you want to run federated searches without special syntax requirements and SPL limitations.
Transparent mode is available in Splunk Cloud Platform 8.2.2107 and Splunk Enterprise 8.3.0. If you want to run federated searches over deployments with lower versions, set those deployments up as standard mode federated providers.
The following table describes the differences between the two modes.
|Category||Standard mode federated search||Transparent mode federated search|
A single deployment can have multiple standard mode federated provider definitions. For example, you might set up different standard mode federated provider definitions for different application contexts.
|Requires federated provider definition only.|
A single deployment can only have one transparent mode federated provider definition.
|Knowledge objects applied to searches||Can use either of these options:
||Must use local knowledge objects from the federated search head.|
|Security||When Local Knowledge Objects is enabled, access to data on the federated provider is determined by role-based security settings on the federated search head. When Local Knowledge Objects is disabled, access to data on the federated provider is determined by role-based settings on the federated provider service account. You can additionally give your users selective role-based access to federated indexes, which restricts the datasets they can search on the federated provider.||Access to data on the federated provider is determined by role-based security settings on the federated search head.|
|When a search runs as a federated search||Searches run as federated searches only when you use federated search syntax to search datasets on standard mode federated providers.||When your local instance is connected to a transparent mode federated provider, all of your searches run as federated searches over that provider. This can lead to an overall decrease in search performance if there are latency issues with the transparent mode federated provider.|
|Special search processing language (SPL) syntax required?||Yes||No|
|SPL limitations||Does not support:
||Does not support |
|Dataset availability||You can search events index and saved search datasets on a federated provider.||You can search events and metrics indexes on a federated provider.|
Enable transparent mode
Transparent mode is disabled by default. If you want to use transparent mode with your federated providers, you must enable transparent mode on your local Splunk platform deployment.
Enable transparent mode for Splunk Cloud Platform
If your local deployment uses Splunk Cloud Platform, contact Splunk Support to enable transparent mode for the deployment. If you have a support contract, file a new case using the Splunk Support Portal at Support and Services. Otherwise, contact Splunk Customer Support.
Federated search security and service accounts
As you prepare to run federated searches, consider the security of the data on the remote Splunk platform deployments that you are setting up as federated providers. Your users' access to that remote data depends on how you will set up those provider definitions. It also depends on how you define the service accounts for each of those federated providers.
A service account is a special user account that you define on a remote Splunk platform deployment before you create its federated provider definition. The service account facilitates the communication between its federated provider and your local Splunk platform deployment. The service account also determines the access that your users have to federated provider data.
If you will be defining a standard mode federated provider, you can set up additional data restrictions on the access your users have to specific federated indexes, ensuring that each user sees only the datasets they are empowered to see.
Defining federated providers
To run a federated search you must create federated provider definitions for the remote Splunk platform deployments that you intend to search. When you define a federated provider you determine whether it will run in standard or transparent mode. See Define a federated provider.
Creating federated indexes
If the federated provider you define uses standard mode, you must create federated indexes for that provider. Each federated index you create is mapped to a specific dataset that you plan to search with federated searches. See Create a federated index.
Running federated searches
For information about constructing and running federated searches, see Run federated searches.
Migrate from hybrid search to federated search
This documentation applies to the following versions of Splunk Cloud Platform™: 8.2.2107