Splunk® Enterprise

Forwarding Data

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Types of forwarders

There are three types of forwarders:

  • The universal forwarder is a streamlined, dedicated version of Splunk Enterprise that contains only the essential components needed to forward data to receivers.
  • A heavy forwarder is a full Splunk Enterprise instance, with some features disabled to achieve a smaller footprint.
  • A light forwarder is also a full Splunk Enterprise instance, with most features disabled to achieve as small a footprint as possible. The light forwarder has been deprecated as of Splunk Enterprise version 6.0. The universal forwarder supersedes the light forwarder for nearly all purposes and represents the best tool for forwarding data to indexers.

For a list of all deprecated features, see "Deprecated features" in the Release Notes.

The universal forwarder

The universal forwarder can gather data from a variety of inputs and forward the data to a Splunk Enterprise server for indexing and searching. It can also forward data to another forwarder as an intermediate step before sending the data onwards to an indexer.

The sole purpose of the universal forwarder is to forward data. Unlike a full Splunk Enterprise instance, you cannot use the universal forwarder to index or search data. To achieve higher performance and a lighter footprint, it has several limitations:

  • The universal forwarder has no searching, indexing, or alerting capability.
  • The universal forwarder does not parse data. You can not use it to route data based on its contents.
  • Unlike full Splunk Enterprise, the universal forwarder does not include a bundled version of Python.

The universal forwarder is a separately downloadable piece of software. Unlike the heavy and light forwarders, you do not enable it from a full Splunk Enterprise instance.

To learn how to download, install, and deploy a universal forwarder, see "Install the universal forwarder software."

For more detail on universal forwarder capabilities, see "The universal forwarder".

Heavy and light forwarders

While the universal forwarder is the preferred way to forward data, you might need to use heavy or light forwarders as well. Unlike the universal forwarder, both heavy and light forwarders are full Splunk Enterprise instances with certain features disabled. Heavy and light forwarders differ in capability and the corresponding size of their resource footprints.

A heavy forwarder (sometimes referred to as a "regular forwarder") has a smaller footprint than an indexer but retains most of the capability, except that it cannot perform distributed searches. Much of its default functionality, such as Splunk Web, can be disabled, if necessary, to reduce the size of its footprint. A heavy forwarder parses data before forwarding it and can route data based on criteria such as source or type of event.

One key advantage of the heavy forwarder is that it can index data locally, as well as forward data to another Splunk Enterprise instance. You must activate this feature. See "Configure forwarders with outputs.conf" in this manual for details.

A light forwarder has a smaller footprint with much more limited functionality. It forwards only unparsed data. Starting with version 4.2, it has been superseded by the universal forwarder, which provides very similar functionality in a smaller footprint. The light forwarder continues to be available mainly to meet legacy needs. We recommend that you always use the universal forwarder to forward unparsed data. When you install a universal forwarder, the installer lets you migrate checkpoint settings from any (version 4.0 or greater) light forwarder that resides on the same machine. See "The universal forwarder" for a more detailed comparison of the universal and light forwarders.

For detailed information on the capabilities of heavy and light forwarders, see "Heavy and light forwarder capabilities."

To learn how to enable and deploy a heavy or light forwarder, see "Deploy a heavy or light forwarder."

Forwarder comparison

This table summarizes the similarities and differences among the three types of forwarders:

Features and capabilities Universal forwarder Light forwarder Heavy forwarder
Type of Splunk Enterprise instance Dedicated executable Full Splunk Enterprise, with most features disabled Full Splunk Enterprise, with some features disabled
Footprint (memory, CPU load) Smallest Small Medium-to-large (depending on enabled features)
Bundles Python? No Yes Yes
Handles data inputs? All types (but scripted inputs might require Python installation) All types All types
Forwards to Splunk Enterprise? Yes Yes Yes
Forwards to 3rd party systems? Yes Yes Yes
Serves as intermediate forwarder? Yes Yes Yes
Indexer acknowledgment (guaranteed delivery)? Optional Optional (version 4.2+) Optional (version 4.2+)
Load balancing? Yes Yes Yes
Data cloning? Yes Yes Yes
Per-event filtering? No No Yes
Event routing? No No Yes
Event parsing? No No Yes
Local indexing? No No Optional, by setting indexAndForward attribute in outputs.conf
Searching/alerting? No No Optional
Splunk Web? No No Optional

For detailed information on specific capabilities, see the rest of this topic, as well as the other forwarding topics in the manual.

Types of forwarder data

Forwarders can transmit three types of data:

  • Raw
  • Unparsed
  • Parsed

The type of data a forwarder can send depends on the type of forwarder it is, as well as how you configure it. Universal forwarders and light forwarders can send raw or unparsed data. Heavy forwarders can send raw or parsed data.

With raw data, the forwarder sends the data stream as raw TCP. it does not convert the data into the Splunk communications format. The forwarder collects the data and forwards it on. This is particularly useful for sending data to a non-Splunk system.

With unparsed data, a universal forwarder performs minimal processing. It does not examine the data stream, but it does tag the stream with metadata to identify source, source type, and host. It also divides the data stream into 64-kilobyte blocks and performs some rudimentary timestamping on the stream, for use by the receiving indexer in case the events themselves have no discernible timestamps. The universal forwarder does not identify, examine, or tag individual events.

With parsed data, a heavy forwarder breaks the data into individual events, which it tags and then forwards to a Splunk Enterprise indexer. It can also examine the events. Because the data has been parsed, the forwarder can perform conditional routing based on event data, such as field values.

The parsed and unparsed formats are both referred to as cooked data, to distinguish them from raw data. By default, forwarders send cooked data — in the universal forwarder's case, unparsed data, and in the heavy forwarder's case, parsed data. To send raw data instead, set the sendCookedData=false attribute/value pair in outputs.conf.

Forwarders and indexes

Forwarders forward and route data on an index-by-index basis. By default, they forward all external data, as well as data for the _audit internal index. In some cases, they also forward data for the _internal internal index. You can change this behavior as necessary. For details, see "Filter data by target index".

PREVIOUS
About forwarding and receiving
  NEXT
The universal forwarder

This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters