Getting Data In

 


Use forwarders

NOTE - Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Use forwarders

Forwarders are lightweight Splunk instances, whose main purpose is to consume data and forward it on to Splunk indexers for further processing. They require minimal resources and have little impact on performance, so they can usually reside on the machines where the data originates.

For example, say you have a number of Apache servers generating data that you want to search centrally. You can install a Splunk indexer on its own Linux machine and then set up forwarders on the Apache machines. The forwarders can take the Apache data and send it on to the Splunk indexer, which then consolidates and indexes it and makes it available for searching. Because of their light footprint, the forwarders won't affect the performance of the Apache servers.

Similarly, you can install forwarders on your employees' Windows desktops. These can send logs and other data to a central Splunk instance, where you can view the data as a whole to track malware or other issues.

What forwarders do

You can use forwarders to get data from remote machines. They represent a much more robust solution than raw network feeds, with their capabilities for:

  • Tagging of metadata (source, sourcetype, and host)
  • Configurable buffering
  • Data compression
  • SSL security
  • Use of any available network ports
  • Running scripted inputs locally

Forwarders consume data in the same way as any other Splunk instance. They can handle exactly the same types of data as a Splunk indexer. The difference is that they usually do not index the data themselves. Instead, they just get the data and send it on to a central Splunk indexer, which does the indexing and searching. A single indexer can index data coming from many forwarders. For detailed information on forwarders, see the "Forwarding data" section of the Distributed Deployment manual.

In most Splunk deployments, forwarders serve as the primary consumers of data. It's only in single-machine deployments that the Splunk indexer is likely to also be the main data consumer. In a large Splunk deployment, you might have hundreds or even thousands of forwarders consuming data and forwarding it on to a group of indexers for consolidation.

How to configure forwarder inputs

As lightweight instances of Splunk, forwarders, by design, have limited capabilities. For example, most forwarders do not include Splunk Web, so you do not have direct access to Splunk Manager for setting up the forwarder's data inputs. Here are the main ways that you can configure a forwarder's data inputs:

  • Specify inputs during initial deployment. For Windows forwarders, you can specify common inputs during the installation process itself. For *nix forwarders, you can specify inputs directly after installation.
  • Use the CLI.
  • Edit inputs.conf.
  • Deploy an app containing the desired inputs.
  • Use Splunk Web on a full Splunk test instance to configure the inputs and then distribute the resulting inputs.conf file to the forwarder itself.

For more information

For detailed information on forwarders, including use cases, typical topologies, and configurations, see "About forwarding and receiving" in the Distributed Deployment manual.

For details on forwarder deployment, including how to use the deployment server to simplify distribution of configuration files and apps to multiple forwarders, see "Universal forwarder deployment overview" in the Distributed Deployment manual.

For information on using forwarders for monitoring of remote Windows data, see "Considerations for deciding how to monitor remote Windows data".

This documentation applies to the following versions of Splunk: 4.2 , 4.2.1 , 4.2.2 , 4.2.3 , 4.2.4 , 4.2.5 , 4.3 , 4.3.1 , 4.3.2 , 4.3.3 , 4.3.4 , 4.3.5 , 4.3.6 , 4.3.7 , 5.0 , 5.0.1 , 5.0.2 , 5.0.3 , 5.0.4 , 5.0.5 , 5.0.6 , 5.0.7 , 5.0.8 , 5.0.9 View the Article History for its revisions.


Comments

a link to download the forwarder would be helpful.

Azabbe
April 27, 2011

You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!