Splunk® Enterprise

Distributed Deployment Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Scale your deployment: Splunk Enterprise components

To accommodate your deployment topology and performance requirements, you can allocate the different Splunk Enterprise roles, such as data input and indexing, to separate Splunk Enterprise instances. For example, you can have instances that just gather data inputs, which they then forward to another, central instance for indexing. Or you can distribute indexing across several instances that coordinate with a separate instance that processes all search requests. To facilitate the distribution of roles, Splunk Enterprise can be configured into a range of separate component types, each mapping to one or more of the roles. You create most components by enabling or disabling specific functions of the full instance.

These are the component types available for use in a distributed environment:

All components are variations of the full Splunk Enterprise instance, with certain features either enabled or disabled, except for the universal forwarder, which is its own executable.

Indexers

The indexer is the Splunk Enterprise component that creates and manages indexes. The primary functions of an indexer are:

  • Indexing incoming data.
  • Searching the indexed data.

In single-machine deployments consisting of just one Splunk Enterprise instance, the indexer also handles the data input and search management functions.

For larger-scale needs, indexing is split out from the data input function and sometimes from the search management function as well. In these larger, distributed deployments, the indexer might reside on its own machine and handle only indexing (usually along with parsing), along with searching of its indexed data. In those cases, other Splunk Enterprise components take over the non-indexing/searching roles. Forwarders consume the data, indexers index and search the data, and search heads coordinate searches across the set of indexers.

For information on indexers, see the Managing Indexers and Clusters of Indexers manual, starting with the topic "About indexes and indexers".

Forwarders

One role that's typically split off from the indexer is the data input function. For instance, you might have a group of Windows and Linux machines generating data that needs to go to a central Splunk Enterprise indexer for consolidation. Usually the best way to do this is to install a lightweight instance of Splunk Enterprise, known as a forwarder, on each of the data-generating machines. These forwarders manage the data input and send the resulting data streams across the network to a Splunk Enterprise indexer, which resides on its own machine. There are two types of forwarders:

  • Universal forwarders. These have a very light footprint and forward only unparsed data.
  • Heavy forwarders. These have a larger footprint but can parse, and even index, data before forwarding it.

For information on forwarders, start with the topic "About forwarding and receiving" in the Forwarding Data manual.

Search heads

In situations where you have a large amount of indexed data and numerous users concurrently searching on it, it can make sense to distribute the indexing and search retrieval load across several indexers, while delegating the search management and presentation functions to a separate machine. In this type of scenario, known as distributed search, one or more Splunk Enterprise components called search heads distribute search requests across multiple indexers.

For information on search heads, see "About distributed search" in the Distributed Search manual.

Deployment server

To update a distributed deployment, you can use the Splunk Enterprise deployment server. The deployment server lets you push out configurations and content to sets of Splunk Enterprise instances (referred to, in this context, as deployment clients), grouped according to any useful criteria, such as OS, machine type, application area, location, and so on. The deployment clients are usually forwarders or indexers. For example, once you've made and tested an updated configuration on a local Linux forwarder, you can push the changes to all the Linux forwarders in your deployment.

The deployment server can cohabit a Splunk Enterprise instance with another Splunk Enterprise component, either a search head or an indexer, if your deployment is small (less than around 50 deployment clients). It should run on its own Splunk Enterprise instance in larger deployments. For more information, see "Estimate deployment server performance" in the Updating Splunk Enterprise Instances manual.

For detailed information on the deployment server, see "About deployment server" in the Updating Splunk Enterprise Instances manual.

Where to go next

While the fundamental issues of indexing and event processing remain the same no matter what the size or nature of your distributed deployment, it is important to take into account deployment needs when planning your indexing strategy. To do that effectively, you must also understand how components map to Splunk Enterprise roles.

For information on hardware requirements for scaling your deployment, see the Capacity Planning manual.

PREVIOUS
How data moves through Splunk Enterprise: the data pipeline
  NEXT
Components and roles

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters