Splunk® Enterprise

Forwarding Data

Download manual as PDF

Download topic as PDF

Types of forwarders

There are three types of forwarders:

  • The universal forwarder contains only the components that are necessary to forward data. Learn more about the universal forwarder in the Universal Forwarder manual.
  • A heavy forwarder is a full Splunk Enterprise instance that can index, search, and change data as well as forward it. The heavy forwarder has some features disabled to reduce system resource usage.
  • A light forwarder is also a full Splunk Enterprise instance, with more features disabled to achieve as small a resource footprint as possible. The light forwarder has been deprecated as of Splunk Enterprise version 6.0. The universal forwarder supersedes the light forwarder for nearly all purposes and represents the best tool for sending data to indexers.

The universal forwarder

The sole purpose of the universal forwarder is to forward data. Unlike a full Splunk instance, you cannot use the universal forwarder to index or search data. To achieve higher performance and a lighter footprint, it has several limitations:

  • The universal forwarder cannot search, index, or produce alerts with data.
  • The universal forwarder does not parse data. You cannot use it to route data to different Splunk indexers based on its contents.
  • Unlike full Splunk Enterprise, the universal forwarder does not include a bundled version of Python.

The universal forwarder can get data from a variety of inputs and forward the data to a Splunk deployment for indexing and searching. It can also forward data to another forwarder as an intermediate step before sending the data onward to an indexer.

The universal forwarder is a separately downloadable piece of software. Unlike the heavy and light forwarders, you do not enable it from a full Splunk Enterprise instance. Learn more about the universal forwarder in the Universal Forwarder manual.

To learn how to download, install, and deploy a universal forwarder, see Install the universal forwarder software in the Universal Forwarder manual.

Heavy and light forwarders

While the universal forwarder is the preferred way to forward data, you might need to use heavy or light forwarders if you need to analyze or make changes to the data before you forward it, or you need to control where the data goes based on its contents. Unlike the universal forwarder, both heavy and light forwarders are full Splunk Enterprise instances with certain features disabled. Heavy and light forwarders differ in capability and the corresponding size of their resource footprints.

A heavy forwarder (sometimes referred to as a "regular forwarder") has a smaller footprint than an indexer but retains most of the capability, except that it cannot perform distributed searches. Some of its default functionality, such as Splunk Web, can be disabled, if necessary, to reduce the size of its footprint. A heavy forwarder parses data before forwarding it and can route data based on criteria such as source or type of event.

One key advantage of the heavy forwarder is that it can index data locally, as well as forward data to another Splunk instance. You must activate this feature. See Configure forwarders with outputs.conf in this manual for details.

A light forwarder has a smaller footprint with much more limited functionality. It forwards only unparsed data. The universal forwarder, which provides very similar functionality, supersedes it. The light forwarder has been deprecated but continues to be available mainly to meet legacy needs.

When you install a universal forwarder, you can migrate checkpoint settings from any (version 4.0 or greater) light forwarder that resides on the same host. See About the universal forwarder in the Universal Forwarder manual for a more detailed comparison of universal and light forwarders.

For detailed information on the capabilities of heavy and light forwarders, see Heavy and light forwarder capabilities in this manual.

Forwarder comparison

This table summarizes the similarities and differences among the three types of forwarders:

Features and capabilities Universal forwarder Light forwarder Heavy forwarder
Type of Splunk Enterprise instance Dedicated executable Full Splunk Enterprise, with most features disabled Full Splunk Enterprise, with some features disabled
Footprint (memory, CPU load) Smallest Small Medium-to-large (depending on enabled features)
Bundles Python? No Yes Yes
Handles data inputs? All types (but scripted inputs might require Python installation) All types All types
Forwards to Splunk Enterprise? Yes Yes Yes
Forwards to 3rd party systems? Yes Yes Yes
Serves as intermediate forwarder? Yes Yes Yes
Indexer acknowledgment (guaranteed delivery)? Optional Optional (version 4.2 and later) Optional (version 4.2 and later)
Load balancing? Yes Yes Yes
Data cloning? Yes Yes Yes
Per-event filtering? No No Yes
Event routing? No No Yes
Event parsing? Sometimes No Yes
Local indexing? No No Optional, by setting indexAndForward attribute in outputs.conf
Searching/alerting? No No Optional
Splunk Web? No No Optional

For detailed information on specific capabilities, see the rest of this topic, as well as the other forwarding topics in the manual.

Types of forwarder data

Forwarders can transmit three types of data:

  • Raw
  • Unparsed
  • Parsed

The type of data a forwarder can send depends on the type of forwarder it is, as well as how you configure it. Universal forwarders and light forwarders can send raw or unparsed data. Heavy forwarders can send raw or parsed data.

With raw data, the forwarder sends the data unaltered over a TCP stream. it does not convert the data into the Splunk communications format. The forwarder collects the data and sends it on. This is particularly useful for sending data to a non-Splunk system.

With unparsed data, a universal forwarder performs minimal processing. It does not examine the data stream, but it does tag the stream with metadata to identify source, source type, and host. It also divides the data stream into 64-kilobyte blocks and performs some rudimentary timestamping on the stream that the receiving indexer can use in case the events themselves have no discernible timestamps. The universal forwarder does not identify, examine, or tag individual events except when you configure it to parse files with structure data (such as comma-separated value files.)

With parsed data, a heavy forwarder breaks the data into individual events, which it tags and then forwards to a Splunk indexer. It can also examine the events. Because the data has been parsed, the forwarder can perform conditional routing based on event data, such as field values.

The parsed and unparsed formats are both referred to as cooked data, to distinguish them from raw data. By default, forwarders send cooked data (universal forwarders send unparsed data and heavy forwarders send parsed data.) To send raw data instead, set the sendCookedData=false attribute/value pair in outputs.conf.

Forwarders and indexes

Forwarders forward and route data on an index-by-index basis. By default, they forward all external data, as well as data for the _audit internal index. In some cases, they also forward data for the _internal internal index. You can change this behavior as necessary. For details, see Filter data by target index.

PREVIOUS
About forwarding and receiving
  NEXT
Forwarder deployment topologies

This documentation applies to the following versions of Splunk® Enterprise: 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 8.0.0


Comments

Hi Nathans,

You could send your ASA logs to one UDP port number and your IOS switch logs to another on your Splunk instance, then set the source types for each port that Splunk listens to on the instance. That would likely be the fastest way to separate the two log types. You wouldn't even need a heavy forwarder to do it.

Alternatives to this are setting source type based on host name, source, or the event data itself. You can learn how to set that up by reading our chapter on source types in the Getting Data In manual.

Malmoore, Splunker
November 8, 2018

I want to send multiple data sources (Cisco ASA and Cisco Switch logs) to a single Heavy Forwarder (HF). Right now I am sending both types of Cisco logs directly to my single Splunk instance via UDP 514. Unfortunately, they are both appearing under the 'cisco:asa' sourcetype, which is not what I want to see. I do see each Cisco device separted based on 'host' but I also want them seperated with Cisco ASA having the 'cisco:asa' sourcetype and Cisco Switch having the 'cisco:ios' sourcetype. From my perspective an HF would allow me to make a distinction between the two types of logs via separate indexes and/or source types. Should I go with the HF option?

Nathans
September 9, 2018

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters