Distributed Deployment Manual

 


Forward data

Forwarder deployment topologies

NOTE - Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Forwarder deployment topologies

You can deploy Splunk forwarders in a wide variety of scenarios. This topic provides an overview of some of the most useful types of topologies that you can create with forwarders. For detailed information on how to configure various deployment topologies, refer to the topics in the section "Use the forwarder to create deployment topologies".

Data consolidation

Data consolidation is one of the most common topologies, with multiple forwarders sending data to a single Splunk server. The scenario typically involves universal forwarders forwarding unparsed data from workstations or production non-Splunk servers to a central Splunk server for consolidation and indexing. With their lighter footprint, universal forwarders have minimal impact on the performance of the systems they reside on. In other scenarios, heavy forwarders can send parsed data to a central Splunk indexer.

Here, three universal forwarders are sending data to a single Splunk indexer:

30 admin13 forwardreceive-dataforward.jpg

For more information on data consolidation, read "Consolidate data from multiple machines".

Load balancing

Load balancing simplifies the process of distributing data across several Splunk indexers to handle considerations such as high data volume, horizontal scaling for enhanced search performance, and fault tolerance. In load balancing, the forwarder routes data sequentially to different indexers at specified intervals.

Splunk forwarders perform automatic load balancing, in which the forwarder switches receivers at set time intervals. If parsing is turned on (for a heavy forwarder), the switching will occur at event boundaries.

In this diagram, three universal forwarders are each performing load balancing between two indexers:

30 admin13 forwardreceive-balance.jpg

For more information on load balancing, read "Set up load balancing".

Routing and filtering

In data routing, a forwarder routes events to specific Splunk or third-party servers, based on criteria such as source, source type, or patterns in the events themselves. Routing at the event level requires a heavy forwarder.

A forwarder can also filter and route events to specific queues, or discard them altogether by routing to the null queue.

Here, a heavy forwarder routes data to three Splunk indexers based on event patterns:

30 admin13 forwardreceive-datarouting.jpg

For more information on routing and filtering, read "Route and filter data".

Forwarding to non-Splunk systems

You can send raw data to a third-party system such as a syslog aggregator. You can combine this with data routing, sending some data to a non-Splunk system and other data to one or more Splunk servers.

Here, three forwarders are routing data to two Splunk servers and a non-Splunk system:

30 admin13 forwardreceive-thirdparty.jpg

For more information on forwarding to non-Splunk systems, read "Forward data to third-party systems".

Intermediate forwarding

To handle some advanced use cases, you might want to insert an intermediate forwarder between a group of forwarders and the indexer. In this type of scenario, the end-point forwarders send data to a consolidating forwarder, which then forwards the data on to an indexer, usually after indexing it locally.

Typical use cases are situations where you need an intermediate index, either for "store-and-forward" requirements or to enable localized searching. (In this case, you would need to use a heavy forwarder.) You can also use an intermediate forwarder if you have some need to limit access to the indexer machine; for instance, for security reasons.

To enable intermediate forwarding, you need to configure the forwarder as a both a forwarder and a receiver. For information on how to configure a receiver, read "Enable a receiver".

This documentation applies to the following versions of Splunk: 4.2 , 4.2.1 , 4.2.2 , 4.2.3 , 4.2.4 , 4.2.5 , 4.3 , 4.3.1 , 4.3.2 , 4.3.3 , 4.3.4 , 4.3.5 , 4.3.6 , 4.3.7 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!