Set up forwarding

Set up forwarding

This topic discusses the options you have when configuring Splunk forwarders. You have the following preconfigured forwarder choices:

• Splunk forwarder
• Splunk light forwarder

If you enable the Splunk forwarder (but not the light forwarder), you also have the option to store a local copy of the indexed data on the forwarding host.

Read this before you enable Splunk forwarder or light forwarder

• If you are configuring forwarding and receiving, your Splunk receiver must be running the same (or later) version of Splunk as your forwarders. A 4.x receiver can receive data from a 3.3.x forwarder, but a 3.3.x receiver cannot receive data from a 4.x forwarder.
• You cannot use round robin data balancing in conjunction with the light forwarder because the data is not parsed (cooked) before being sent--events may be split into parts before reaching the receiver, resulting in partial events. You can, however, use automatic load balancing.
• Splunk Web is turned off in the light forwarder to reduce the footprint of Splunk on the forwarding host. Therefore, if you want to use Splunk Web to configure your forwarding Splunk instance, do this before you enable the forwarder application. After you enable the forwarder application, you can only configure your forwarder via the Splunk CLI or by editing its configuration files.
• If you enable the light forwarder on a host to which you have been distributing searches via distributed search, the searches will no longer work on that host; indexes are disabled even if they contain data.
• You must configure a receiver before setting up forwarding. This way, the Splunk receiving host is prepared for the forwarded data. Then, configure your forwarder(s).
• If you're defining a source type based on source:: in props.conf you must set this on the forwarding side, it will not take effect if set on the receiving side.
• By default, Splunk will use an 'Enterprise Trial' license when it is initially installed. If you are enabling any of the forwarding apps, you should also apply either the forwarder license or the free license to avoid any subsequent license issues. Full instructions on how to do this can be found here.

Forwarding OS app data from one OS to a different OS

If you're using the OS specific apps such as the *NIX app or the Windows app, and forwarding from one OS to a different OS (for example, from Windows to a Linux indexer), you should install the OS-specific App for the forwarder's OS on the indexer in order to view the data. For example, if you're forwarding from Windows to Linux, you should install the Splunk for Windows App on the Linux receiver. Once you've downloaded the relevant OS-specific App, rename or delete the inputs.conf for that App (for the Windows App, it's $SPLUNK_HOME/etc/apps/windows/default/inputs.conf before enabling the App to ensure its default inputs are not added to your indexer.

What's the Splunk forwarder?

The Splunk forwarder is a version of Splunk that allows you to send data to a central Splunk indexer or group of indexers. The distributed search module is disabled for the Splunk forwarder (in $SPLUNK_HOME/etc/apps/SplunkForwarder/default/default-mode.conf):

[pipeline:distributedSearch]
disabled = true

All other functions and modules remain enabled.

For a detailed view of the exact configuration, you can look at the configuration files for the SplunkForwarder application in $SPLUNK_HOME/etc/apps/SplunkForwarder/default, where SPLUNK_HOME is the directory into which you installed Splunk.

What's the Splunk light forwarder?

The Splunk light forwarder is a lightweight version of Splunk that can monitor local log files and directories, collect Windows event logs and use scripted inputs (including local WMI and registry data sources on Windows). To cut down on overhead, many other features of the standard Splunk server are disabled.

Specifically, the Splunk light forwarder:

• Disables event signing and checking if the disk is full (/$SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/default-mode.conf)
• Disables the file system change monitor (fschange) (/$SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/inputs.conf)
• Limits internal data inputs to splunkd and metrics logs only, and makes sure these are forwarded ($SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/inputs.conf)
• Disables all indexing ($SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/indexes.conf)
• Does not use transforms.conf and does not fully parse incoming data, but the CHARSET, CHECK_FOR_HEADER, NO_BINARY_CHECK, PREFIX_SOURCETYPE, and sourcetype properties from props.conf are used.
• Disables the Splunk Web interface ($SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/web.conf )
• Limits throughput to 256KBps (/etc/apps/SplunkLightForwarder/default/limits.conf)
• Disables the following modules (in $SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/default-mode.conf):

[pipeline:indexerPipe]
disabled_processors= indexandforward, diskusage, signing,tcp-output-generic-processor, 
syslog-output-generic-processor, http-output-generic-processor, stream-output-processor

[pipeline:distributedDeployment]
disabled = true

[pipeline:distributedSearch]
disabled = true

[pipeline:fifo]
disabled = true

[pipeline:merging]
disabled = true

[pipeline:typing]
disabled = true

[pipeline:udp]
disabled = true

[pipeline:tcp]
disabled = true

[pipeline:syslogfifo]
disabled = true

[pipeline:syslogudp]
disabled = true

[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut 

These modules are the deployment server (not the deployment client), distributed search, and from named pipes / FIFOs, and direct input from network ports.

Configure and enable forwarding in Splunk Web

When you enable forwarding (not lightweight forwarding) in Splunk Web, you can choose to have a copy of the indexed data stored locally on this host as well as forwarded to each receiver. To do this:

1. Log into Splunk Web.

2. Click Manager in the upper right corner.

3. Click Forwarding and receiving > Forwarding defaults.

4. Select Yes to store and maintain a local copy of the indexed data on this host. This setting does not take effect if you subsequently enable lightweight forwarding.

To enable Splunk forwarder or light forwarder via Splunk Web:

1. Log into Splunk Web.

2. Click Manager in the upper right corner.

3. Click Forwarding and receiving.

4. Choose a forwarding option:

• To forward pre-processed, transformed data -- already split into events -- click Configure forwarding to hosts and provide a comma-separated list of one or more hosts using the format host:port or IP:port. Splunk will send a copy of all events to each host you specify. You must have configured receiving on each host to which you wish to forward.
• To forward minimally pre-processed data, click Enable lightweight forwarding. You must use the CLI to further configure this host for forwarding, so be sure you understand what you're doing.

Important: Remember, if you enable Splunk light forwarder, Splunk Web will subsequently be unreachable--you will only have access to configure this Splunk instance via the CLI.
Note: Don't forget to install either the free, or forwarder license.

Configure a Splunk instance as a forwarder via the CLI

To enable a Splunk forwarder or light forwarder via the CLI:

./splunk enable app [SplunkForwarder|SplunkLightForwarder] -auth <username>:<password>

To determine what type of forwarder a particular Splunk instance is, run the following command:

./splunk display app -auth <username>:<password>

Note: Don't forget to install either the free, or forwarder license.
Note: If you are running Splunk with a free license, you do not have to provide a username and password to enable apps.

Disable a Splunk forwarder or light forwarder via the CLI

To disable Splunk forwarder or light forwarder via the CLI:

./splunk disable app [SplunkForwarder|SplunkLightForwarder] -auth <username>:<password>

Note: If you are running Splunk with a Free license, you do not have to provide a username and password.

Add or remove existing forwarders via the CLI

Enable forwarding from the Splunk CLI. Navigate to your $SPLUNK_HOME/bin directory on the forwarding server and log in to the CLI.

./splunk login
Splunk username: admin
Password: 

To add a forwarder:

# ./splunk add forward-server <host:port> -auth admin:changeme

where <host:port> are the hostname and port of the Splunk server to which this forwarder or light forwarder should send data.

To remove an existing forwarder:

# ./splunk remove forward-server <host:port> -auth admin:changeme

where <host:port> are the hostname and port of the Splunk server to which this forwarder or light forwarder is currently sending data.

Note: Although this command disables the forwarding activity, this machine will still be configured as a Splunk forwarder or light forwarder.

• Was this topic useful?
• Post a comment