Types of forwarders

Types of forwarders

There are three types of forwarders:

• The universal forwarder is a streamlined, dedicated version of Splunk that contains only the essential components needed to forward data to receivers.
• A heavy forwarder is a full Splunk instance, with some features disabled to achieve a smaller footprint.
• A light forwarder is also a full Splunk instance, with most features disabled to achieve as small a footprint as possible. The universal forwarder, with its even smaller footprint yet similar functionality, supersedes the light forwarder for nearly all purposes.

In nearly all respects, the universal forwarder represents the best tool for forwarding data to indexers. Its main limitation is that it forwards only unparsed data, as described later in this topic. Therefore, you cannot use it to route data based on event contents. For that, you must use a heavy forwarder. You also cannot index data locally on a universal forwarder; only a heavy forwarder can index and forward.

The universal forwarder

The universal forwarder is Splunk's new lightweight forwarder. You use it to gather data from a variety of inputs and forward the data to a Splunk server for indexing and searching. You can also forward data to another forwarder, as an intermediate step before sending the data onwards to an indexer.

The universal forwarder's sole purpose is to forward data. Unlike a full Splunk instance, you cannot use the universal forwarder to index or search data. To achieve higher performance and a lighter footprint, it has several limitations:

• The universal forwarder has no searching, indexing, or alerting capability.
• The universal forwarder does not parse data.
• Unlike full Splunk, the universal forwarder does not include a bundled version of Python.

For details on the universal forwarder's capabilities, see "Introducing the universal forwarder".

Note: The universal forwarder is a separately downloadable piece of software. Unlike the heavy and light forwarders, you do not enable it from a full Splunk instance. To learn how to download, install, and deploy a universal forwarder, see "Universal forwarder deployment overview".

Heavy and light forwarders

While the universal forwarder is generally the preferred way to forward data, you might have reason (legacy-based or otherwise) to use heavy or light forwarders as well. Unlike the universal forwarder, which is an entirely separate, streamlined executable, both heavy and light forwarders are actually full Splunk instances with certain features disabled. Heavy and light forwarders differ in capability and the corresponding size of their footprints.

A heavy forwarder (sometimes referred to as a "regular forwarder") has a smaller footprint than a Splunk indexer but retains most of the capability, except that it lacks the ability to perform distributed searches. Much of its default functionality, such as Splunk Web, can be disabled, if necessary, to reduce the size of its footprint. A heavy forwarder parses data before forwarding it and can route data based on criteria such as source or type of event.

One key advantage of the heavy forwarder is that it can index data locally, as well as forward data to another Splunk instance. You must turn this capability on; it's disabled by default. See "Configure forwarders with outputs.conf" in this manual for details.

A light forwarder has a smaller footprint with much more limited functionality. It forwards only unparsed data. Starting with 4.2, it has been superseded by the universal forwarder, which provides very similar functionality in a smaller footprint. The light forwarder continues to be available mainly to meet any legacy needs. We recommend that you always use the universal forwarder to forward unparsed data. When you install a universal forwarder, the installer gives you the opportunity to migrate checkpoint settings from any (version 4.0 or greater) light forwarder residing on the same machine. See "Introducing the universal forwarder" for a more detailed comparison of the universal and light forwarders.

For detailed information on the capabilities of heavy and light forwarders, see "Heavy and light forwarder capabilities".

To learn how to enable and deploy a heavy or light forwarder, see "Deploy a heavy or light forwarder".

Forwarder comparison

This table summarizes the similarities and differences among the three types of forwarders:

Features and capabilities	Universal forwarder	Light forwarder	Heavy forwarder
Type of Splunk instance	Dedicated executable	Full Splunk, with most features disabled	Full Splunk, with some features disabled
Footprint (memory, CPU load)	Smallest	Small	Medium-to-large (depending on enabled features)
Bundles Python?	No	Yes	Yes
Handles data inputs?	All types (but scripted inputs might require Python installation)	All types	All types
Forwards to Splunk?	Yes	Yes	Yes
Forwards to 3rd party systems?	Yes	Yes	Yes
Serves as intermediate forwarder?	Yes	Yes	Yes
Indexer acknowledgment (guaranteed delivery)?	Optional	Optional (version 4.2+)	Optional (version 4.2+)
Load balancing?	Yes	Yes	Yes
Data cloning?	Yes	Yes	Yes
Per-event filtering?	No	No	Yes
Event routing?	No	No	Yes
Event parsing?	No	No	Yes
Local indexing?	No	No	Optional, by setting indexAndForward attribute in outputs.conf
Searching/alerting?	No	No	Optional
Splunk Web?	No	No	Optional

For detailed information on specific capabilities, see the rest of this topic, as well as the other forwarding topics in the manual.

Types of forwarder data

Forwarders can transmit three types of data:

• Raw
• Unparsed
• Parsed

The type of data a forwarder can send depends on the type of forwarder it is, as well as how you configure it. Universal forwarders and light forwarders can send raw or unparsed data. Heavy forwarders can send raw or parsed data.

With raw data, the data stream is forwarded as raw TCP; it is not converted into Splunk's communications format. The forwarder just collects the data and forwards it on. This is particularly useful for sending data to a non-Splunk system.

With unparsed data, a universal forwarder performs only minimal processing. It does not examine the data stream, but it does tag the entire stream with metadata to identify source, source type, and host. It also divides the data stream into 64K blocks and performs some rudimentary timestamping on the stream, for use by the receiving indexer in case the events themselves have no discernible timestamps. The universal forwarder does not identify, examine, or tag individual events.

With parsed data, a heavy forwarder breaks the data into individual events, which it tags and then forwards to a Splunk indexer. It can also examine the events. Because the data has been parsed, the forwarder can perform conditional routing based on event data, such as field values.

The parsed and unparsed formats are both referred to as cooked data, to distinguish them from raw data. By default, forwarders send cooked data — in the universal forwarder's case, unparsed data, and in the heavy forwarder's case, parsed data. To send raw data instead, set the sendCookedData=false attribute/value pair in outputs.conf.

Forwarders and indexes

Forwarders forward and route data on an index-by-index basis. By default, they forward all external data, as well as data for the _audit internal index. You can change this behavior as necessary. For details, see "Filter data by target index".

• Was this topic useful?
• Post a comment