Splunk® Universal Forwarder

Forwarder Manual

Download manual as PDF

Download topic as PDF

Configure a forwarder to handle multiple pipeline sets

You can configure a forwarder to use multiple processing pipelines to increase forwarding throughput for machines that forward a large amount of data and have more than one core available.

Forwarding pipelines

The universal forwarder can be configured to handle multiple pipeline sets. A pipeline set is an instance of the event processing section of the data pipeline. A universal forwarder with multiple pipeline sets can process multiple events at once, thus increasing forwarder throughput and getting events to indexers faster. The feature that both indexers and forwarders use to handle multiple streams is called index parallelization.

Other than the fact that the forwarders can send more than one stream of data at a time, forwarders with parallelization enabled look just like other forwarders to indexers. As well, indexers process data streams from forwarders with parallelization enabled in the same way that they process forwarders with one data stream.

While it is possible to increase the number of pipelines by any number, you should not increase it to more than two unless you receive instruction to do so by Splunk Professional Services.

When you enable multiple pipeline sets on a forwarder, any throughput-related settings apply to each pipeline set on the forwarder, rather than to the forwarder itself. For example, if you have two pipeline sets enabled on a forwarder, the maximum network throughput default of 256KBps (2 Mbps) is for each pipeline, for a total of 512KBps (4Mbps) for the forwarder.

For more information about how forwarders use parallelization to process more data, see Forwarders and multiple pipeline sets in the Splunk Enterprise Managing Indexers and Clusters of Indexers manual.

Configure a universal forwarder to use multiple pipeline sets

1. At a command or shell prompt on the universal forwarder, change to the local configuration directory:

Unix Windows
cd $SPLUNK_HOME/etc/system/local cd %SPLUNK_HOME%\etc\system\local

2. Create the file server.conf in this directory.

3. Open the server.conf file for editing.

4. Add the following stanza to enable parallelization:

[general]
parallelIngestionPipelines = 2

5. Save the file and close it.

6. Restart the universal forwarder.

PREVIOUS
Configure an intermediate forwarder
  NEXT
Configure forwarding to Splunk Enterprise indexer clusters

This documentation applies to the following versions of Splunk® Universal Forwarder: 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 8.0.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters