Connect forwarders directly to peer nodes

These are the main steps for setting up connections between forwarders and peer nodes, using the traditional method of connecting each forwarder directly to each peer node:

1. Configure the peer nodes to receive data from forwarders.

2. Configure the forwarders to send data to the peer nodes.

3. Enable indexer acknowledgment for each forwarder. This step is required to ensure end-to-end data fidelity. If that is not a requirement for your deployment, you can skip this step.

Once you finish setting up the connection, you must configure the data inputs on the forwarders. See "Configure the forwarder's data inputs".

1. Configure the peer nodes to receive data from forwarders

In order for a peer to receive data from forwarders, you must configure the peer's receiving port. For information on how to configure the receiving port, read "Enable a receiver" in the Forwarding Data manual.

One way to specify the receiving port is to edit the peer's inputs.conf file. You can simplify peer input configuration by deploying a single, identical inputs.conf file across all the peers. The receiving port that you specify in the common copy of inputs.conf will supersede any ports you enable on each individual peer. For details on how to create and deploy a common inputs.conf across all peers, read "Update common peer configurations".

2. Configure the forwarders to send data to the peer nodes

When you set up a forwarder, you specify its receiving peer by providing the peer's IP address and receving port number. For example: 10.10.10.1:9997. You do this in the forwarder's outputs.conf file, as described in "Configure forwarders with outputs.conf" in the Forwarding Data manual. To specify the receiving peer, set the server attribute, like this:

server=10.10.10.1:9997

The receiving port that you specify here is the port that you configured on the peer in step 1.

To set up the forwarder to use load-balancing, so that the data goes to multiple peer nodes in sequence, you configure a load-balanced group of receiving peers. For example, this attribute/value pair in outputs.conf specifies a load-balanced group of three peers:

server=10.10.10.1:9997,10.10.10.2:9997,10.10.10.3:9997

To learn more about configuring load balancing, read "Set up load balancing" in the Forwarding Data manual.

Note: There are several other ways that you can specify a forwarder's receiving peer(s). For example:

• You can specify the receiving peer during universal forwarder deployment (for Windows universal forwarders only), as described in Install a Windows universal forwarder in the Universal Forwarder manual.
• You can specify the receiver with the CLI command add forward-server, as described in Enable a receiver in the Forwarding Data manual.

Both of these methods work by modifying the underlying outputs.conf file. No matter what method you use to specify the receiving peers, you still need to directly edit the underlying outputs.conf file if you want to turn on indexer acknowledgment, as described in the next step.

3. Enable indexer acknowledgment for each forwarder

This step is required to ensure end-to-end data fidelity. If that is not a requirement for your deployment, you can skip this step.

To ensure that the cluster receives and indexes all incoming data, you must turn on indexer acknowledgment for each forwarder.

Caution: Indexer acknowledgement can, under some circumstances, result in duplicate events. To learn about this issue and how to work around it, see Protect against loss of in-flight data in the Forwarding Data manual.

To configure indexer acknowledgment, set the useACK attribute in each forwarder's outputs.conf:

[tcpout:<peer_target_group>]
useACK=true

For detailed information on configuring indexer acknowledgment, read Protect against loss of in-flight data in the Forwarding Data manual.

Caution: For indexer acknowledgment to work properly, the forwarders' wait queues must be configured to the optimal size. For forwarders at version 5.0.4 or above, the system handles this automatically. For earlier version forwarders, follow the instructions in the version of the Protect against loss of in-flight data topic for that forwarder version. Specifically, read the subtopic on adjusting the maxQueueSize setting.

Example: A load-balancing forwarder with indexer acknowledgment

Here is a sample outputs.conf configuration for a forwarder that is using load balancing to send data in sequence to three peers in a cluster. It assumes that each of the peers has previously been configured to use 9997 for its receiving port:

[tcpout]
defaultGroup=my_LB_peers

[tcpout:my_LB_peers]
autoLBFrequency=40
server=10.10.10.1:9997,10.10.10.2:9997,10.10.10.3:9997
useACK=true

The forwarder starts by sending data to one of the peers listed for the server attribute. After 40 seconds, it switches to another peer, and so on. If, at any time, it doesn't receive acknowledgment from the current receiving node, it resends the data, this time to the next available node.