Federated Search for Splunk platform

The following image shows architecture diagrams of federated search in transparent mode (hybrid search replacement), standard mode with a customer managed federated search head, and standard mode with a Splunk Cloud Platform federated search head. The diagram also a legend of terminology and icons used in the diagram for federated search head, federated provider, and federated dataset.

Benefits

Federated Search provides inter-environment search capabilities to support customers who have multiple independent Splunk Enterprise or Splunk Cloud Platform environments.

Federated Search has two operating modes that are useful in different situations.

Standard Mode allows access to specific configured remote datasets from a Federated Provider, allowing for role-based control of access to remote datasets with a high degree of control.

Transparent Mode is designed to support customers migrating from Customer Managed Platform (CMP) environments into Splunk Cloud Platform where there is an existing Hybrid Search Head. Transparent mode provides "transparent" access to datasets in a CMP indexer environment and a Splunk Cloud Platform Federated Provider with no changes to searches required, and existing object and index names are preserved.

In both modes, customers can benefit from centralized access to datasets which may exist in multiple locations. Customers migrating from CMP environments to Splunk Cloud Platform can seamlessly address datasets with the same name across Splunk Cloud Platform and a CMP environment using Transparent Mode. Customers who have multiple Splunk Cloud Platform stacks or Splunk Enterprise environments for historical, functional, or data-sovereignty related reasons can maintain this structure while providing remote access to appropriate specific datasets using Standard Mode.

Federated Search Transparent Mode and Standard Mode both give more options for security and access control when compared with distributed/hybrid search. Federated Providers can have greater control of which Federated Search Heads have access to datasets and can appropriately set resource capacity limits.

Federated Search can simplify environment interconnectivity requirements by utilizing communication between search heads on the splunk management port interface.

By combining environment search interconnectivity using Federated Search with the Splunk Cloud Platform Admin Config Service to manage and synchronize the configurations of multiple environments, customers can significantly simplify the management of multiple Splunk Cloud Platform resources.

Use cases

Use of Federated Search supports a number of existing and new use cases, simplifying architecture that was previously supported by other features and functionality.

Hybrid Search

Hybrid Search is a legacy feature that allows Splunk Enterprise search heads to connect to Splunk Cloud Platform indexers. Existing Hybrid Search Heads should be migrated to Federated Search in Transparent Mode, or if possible, Standard Mode. See Migrate from hybrid search to Federated Search for Splunk in the Splunk Enterprise Federated Search manual.

Distributed search

In some cases, where previously an independent search head was used to aggregate data from many separate deployments, this can now be achieved using Federated Search in Standard Mode. Federated Search also allows for searches to be distributed to and between Splunk Cloud Platform environments, which previously was not possible.

Requirements

Configuration

For the correct operation of Federated Search, a number of configurations must be in place in the deployments in which you intend to distribute and receive federated searches. These configurations and requirements are outlined in the following sections for the Federated Provider, Federated Search Head, and intermediate communication network.

Federated Provider

The Federated Provider is the environment that receives and executes federated searches, returning results back to the Federated Search Head.

Service accounts are required on the Federated Provider for both Standard and Transparent mode searches. Service accounts for each mode have different permissions requirements. See Service accounts and security for Federated Search for Splunk n the Splunk Enterprise Federated Search manual.

Federated Search Head

The Federated Search Head is the search head or Splunk Cloud Platform stack from which Federated Searches are distributed to Federated Providers. Federated Search Heads are configured with a connection to one or more Federated Providers.

When using Standard Mode, Federated Search Heads are also configured with Federated Indexes that can reference a number of different types of remote dataset. These include Indexes, Accelerated Data Models, and Saved Searches. As new indexes and datasets are added in the Federated Provider, customers must add new configurations to the Federated Search Head to enable access.

Version requirements

Minimum version requirements for Splunk Cloud Platform and Splunk Enterprise as both Federated Search Head and Federated provider are described in About Federated Search for Splunk in the Splunk Enterprise Federated Search manual.

Communication

For Federated Search to operate, the Federated Search Head must be able to communicate with the Federated Provider's Remote Search Head on the management port of the Remote Search Head. For Search Head Clusters, this communication must be possible for all search heads in the Federated Provider cluster, or Federated Search Head cluster.

If the federated provider is part of a CMP environment, firewalls must allow communication between Federated Search Heads and CMP Search Heads, through whatever firewalls exist. For Splunk Cloud Platform stacks, customers might need to request appropriate changes using a support request.

Limitations and best practices

Premium Apps

This document does not cover best practices for use of Federated Search with Premium Apps such as Enterprise Security (ES) or Splunk IT Service Intelligence (ITSI). If you require support in this area, contact your account or customer support team.

Search concurrency

When searches are distributed using federated search, remote portions of the search might not execute if the Federated Provider has no remaining search concurrency/capacity. It is important to monitor the search concurrency and capacity on Federated Providers to ensure that resources are available, and to correctly configure resource limits in the service accounts on Federated Providers.

Inter-deployment communication

When using federated search across multiple deployments, the communication infrastructure that supports inter-deployment connectivity is a significant factor. Deployments that are connected using public WAN or private interconnection might be impacted by connection failures, bandwidth limits, and latency.

When possible, schedule and execute searches on infrastructure that is directly connected to data sources. This helps to minimize the number of searches and the amount of data transferred via a Federated Search.

Configuration management

Configuration management across multiple deployments becomes more important when using Federated Search. In both modes, knowledge objects and configurations that exist on the Federated Provider can influence search results.

Customers must synchronize knowledge objects between Federated Search Head and Federated Provider for correct operation of this feature. See Custom knowledge object coordination for standard mode federated providers in the Splunk Enterprise Federated Search manual.

Effectively understanding and modifying or synchronizing configurations on the Federated Provider is important for many implementations. You can accomplish this manually with automated configuration management, and in Splunk Cloud Platform using the Admin Config Service.

In most situations, building simple and effective naming conventions, such as adding prefixes to saved search dataset names, can help to keep configurations organized and understood. Given the increased complexity of managing datasets across multiple environments, this is highly recommended when working with Federated Search.

Mode-specific limitations

On any one Federated Search Head, enable only one federated search mode at a time. Do not enable standard mode and transparent mode at the same time.

Standard mode and transparent mode each have some limitations with respect to supported SPL commands and functionality. For more information, see the Splunk Enterprise or Splunk Cloud Platform documentation.

• For Splunk Cloud Platform, see the two "restrictions" sections of Run federated searches in the Search Manual. Use the version drop-down list on that page to choose the version of the product that you use.
• For Splunk Enterprise, see the two "restrictions" sections of Run federated searches in the Search Manual. Use the version drop-down list on that page to choose the version of the product that you use.