About forwarding and receiving
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
About forwarding and receiving
If you have a number of sources distributed across your environment, you have the option of installing a lighter weight version of Splunk on each source machine and forwarding the data from that machine to one or more central Splunk indexer instance(s).
A Splunk server running on any supported OS platform can forward data to another Splunk instance (as well as to other systems) in real time. This allows data gathered on one Splunk host in a specific environment to be sent to another Splunk instance for indexing and search.
You can also configure Splunk instances to forward data to groups of other Splunk instances, to enable horizontal scaling via clustered indexing. Splunk instances can also clone data to multiple groups of other Splunk instances to provide for data redundancy in high availability environments.
Forwarding and receiving includes all configurations in which one Splunk instance (the forwarder) is sending data to one or more Splunk indexers (the receivers) prior to being indexed. The forwarder can also index data locally.
Older forwarder versions
3.3.x forwarders will work with a 4.x version of Splunk. This is especially useful for large forwarder deployments. You can wait until you are comfortable with your new deployment server configuration before migrating forwarders to 4.x.
Important: Your receiver must be running either the same version or a later version of Splunk as your forwarders. For example, a 4.0 receiver can accept traffic from forwarders running earlier versions. A 3.4 receiver cannot accept connections from a 4.0 forwarder.
Forwarding
Forwarding is the simplest setup for forwarding and receiving. Forwarding refers to any Splunk instance that sends data to another server for indexing.
There are two types of Splunk forwarders, the forwarder, and the light forwarder. The main difference between the two is that the forwarder processes the data before forwarding it, whereas the light forwarder sends unprocessed data to the receiver.
Refer to Set up forwarding for more information about Splunk forwarders.
Routing
With routing enabled, the forwarder matches conditions based on patterns in the events themselves to selectively send some events to one Splunk instance and other events to another instance.
Cloning
In the context of forwarding and receiving, cloning refers specifically to a forwarder sending every event to two or more Splunk instances to provide for data redundancy. It should be noted that this does not guarantee two or more exactly identical indexes; if one of the receivers becomes unavailable, data is only sent to the receivers that are available. This can result in non-identical indexes.
Automatic load balancing and round-robin data balancing
You can designate where the data from a given forwarder is sent based on defining groups of Splunk indexers using either automatic load balancing or data balancing. These configurations support large volumes of data by letting you create target groups of indexing receivers to which the forwarders send data based on rules you define. You can specify how these indexers are chosen by the forwarders, and how the lists of available receivers are managed (as static lists on each forwarder, or via DNS records).
Buffering during load balancing
If a server becomes inaccessible during load balancing, Splunk continues to send events to all accessible servers.
Eventually, Splunk stops trying to send to an unresponsive server, and notes that the server has gone off line. If all servers are inaccessible, Splunk writes to a buffer on the forwarder's side.
Target groups
In addition to being able to send data to a single receiver, forwarders can send to indexers in target groups. Target groups consist of one or more receiving indexers. A given receiver can be part of multiple target groups.
Cloning sends every event to all target groups; routing sends specific events to one target group and different events to other target groups. You can also set up default groups, which receive all the data not sent to target groups. If more than one group is specified, Splunk clones events to all listed default groups.
defaultGroup=<groupname1>,<groupname2>...
Send to third party systems (using syslog or HTTP)
By default, data is sent from a standard Splunk forwarder as cooked' data, meaning that the events have been indexed and are ready for searching when they arrive at the Splunk receiver. However, you can configure Splunk forwarders to send uncooked, or raw data so that third party systems can handle it correctly. In particular, you can specify that data be sent to a syslog aggregator or to an HTTP host.
Security
Any Splunk forwarder can send some or all of its incoming data in real time to other Splunk servers and to other systems via TCP, either in clear text or via SSL.
This documentation applies to the following versions of Splunk: 4.0 , 4.0.1 , 4.0.2 , 4.0.3 , 4.0.4 , 4.0.5 , 4.0.6 , 4.0.7 , 4.0.8 , 4.0.9 , 4.0.10 , 4.0.11 View the Article History for its revisions.