Distributed Deployment Manual

Components and roles

Each segment of the data pipeline directly corresponds to a role that one or more Splunk Enterprise components can perform. For instance, data input is a Splunk Enterprise role. Either an indexer or a forwarder can perform the data input role. For more information on the data pipeline, look here.

How components support the data pipeline

This table correlates the pipeline segments and Splunk Enterprise roles with the components that can perform them:

Data pipeline segment	Role	Components that can perform this role
Data input	Data input	indexer universal forwarder heavy forwarder
Parsing	Parsing	indexer heavy forwarder
Indexing	Indexing	indexer
Search	Search	indexer search head
n/a	Managing distributed updates	deployment server

As the table indicates, some roles can be filled by diffferent components depending on the situation. For instance, data input can be handled by an indexer in single-machine deployments, or by a forwarder in larger deployments.

For more information on components, look here.

Components in action

These are some of the common ways in which Splunk Enterprise functionality is distributed and managed.

Forward data to an indexer

In this deployment scenario, forwarders handle data input, collecting data and send it on to a Splunk Enterprise indexer. Forwarders come in two flavors:

• Universal forwarders. These maintain a small footprint on their host machine. They perform minimal processing on the incoming data streams before forwarding them on to an indexer, also known as the receiver.
• Heavy forwarders. These retain much of the functionality of a full Splunk Enterprise instance. They can parse data before forwarding it to the receiving indexer. (See "How data moves through Splunk Enterprise" for the distinction between parsing and indexing.)

Both types of forwarders tag data with metadata such as host, source, and source type, before forwarding it on to the indexer.

Forwarders allow you to use resources efficiently while processing large quantities or disparate types of data. They also enable a number of interesting deployment topologies, by offering capabilities for load balancing, data filtering, and routing.

For an extended discussion of forwarders, including configuration and detailed use cases, see "About forwarding and receiving" in the Forwarding Data manual.

Search across multiple indexers

In distributed search, Splunk Enterprise instances send search requests to other Splunk Enterprise instances and merge the results back to the user. This is useful for a number of purposes, including horizontal scaling, access control, and managing geo-dispersed data.

The Splunk Enterprise instance that manages search requests is called the search head. The instances that maintain the indexes and perform the actual searching are indexers, called search peers in this context.

For an extended discussion of distributed search, including configuration and detailed use cases, see "About distributed search" in the Distributed Search manual.

Manage distributed updates

When dealing with distributed deployments consisting potentially of many forwarders, indexers, and search heads, the Splunk Enterprise deployment server simplifies the process of configuring and updating Splunk Enterprise components, mainly forwarders and indexers. Using the deployment server, you can group the components (referred to as deployment clients in this context) into server classes, making it possible to push updates based on common characteristics.

A server class is a set of Splunk Enterprise instances that share configurations. Server classes are typically grouped by OS, machine type, application area, location, or other useful criteria. A single deployment client can belong to multiple server classes, so a Linux universal forwarder residing in the UK, for example, might belong to a Linux server class and a UK server class, and receive configuration settings appropriate to each.

For an extended discussion of deployment management, see "About deployment server" in the Updating Splunk Enterprise Instances manual.

For more information

In summary, these are the fundamental components and features of a Splunk Enterprise distributed environment:

• Indexers. See "About indexes and indexers" in the Managing Indexers and Clusters of Indexers manual.
• Forwarders. See "About forwarding and receiving" in the Forwarding Data manual.
• Search heads. See "About distributed search" in the Distributed Search manual.
• Deployment server. See "About deployment server" in the Updating Splunk Enterprise Instances manual.

For guidance on where to configure various Splunk Enterprise settings, see "Configuration parameters and the data pipeline" in the Admin Manual. That topic lists key configuration settings and the data pipeline segments they act upon. If you know which components in your Splunk Enterprise topology handle which segments of the data pipeline, you can use that topic to determine where to configure the various settings. For example, if you use a search head to handle the search segment, you'll need to configure any search-related settings on the search head and not on your indexers.