Enable a receiver

Enable a receiver

To enable forwarding and receiving, you configure both a receiver and a forwarder. The receiver is the Splunk instance receiving the data; the forwarder sends data to the receiver.

Depending on your needs (for example to enable load balancing), you might have multiple receivers for each forwarder. Conversely, a single receiver usually receives data from many forwarders.

The receiver is either a Splunk indexer (the typical case) or another forwarder (referred to as an "intermediate forwarder") configured to receive data from forwarders.

You must set up the receiver first. You can then set up forwarders to send data to that receiver.

Compatibility between forwarders and indexers

These are the compatibility restrictions between versions of forwarders and indexers:

• 4.2+/5.0+ forwarders (universal/light/heavy) are backwards compatible down to 4.2+ indexers. For example, a 4.3 forwarder can send data to a 4.2 indexer but not to a 4.1 indexer.

• Pre-4.2 forwarders are backwards compatible down to 4.0 indexers.

• All indexers are backwards compatible with any forwarder and can receive data from any earlier version forwarder. For example, a 4.2 indexer can receive data from a 4.1 forwarder.

For each app, check Splunkbase for version compatibility.

Note: Splunk recommends that the indexer version should be the same or newer than the version of the forwarders sending to it. Although we strive to ensure backward compatibility, it is not always possible.

Set up receiving

Before enabling a Splunk instance (either an indexer or a forwarder) as a receiver you must, of course, first install it.

You can then enable receiving on a Splunk instance through Splunk Web, the CLI, or the inputs.conf configuration file.

Set up receiving with Splunk Web

Use Splunk Manager to set up a receiver:

1. Log into Splunk Web as admin on the server that will be receiving data from a forwarder.

2. Click the Manager link in the upper right corner.

3. Select Forwarding and receiving in the Data area.

4. Click Add new in the Receive data section.

5. Specify which TCP port you want the receiver to listen on (the listening port, also known as the receiving port). For example, if you enter "9997," the receiver will receive data on port 9997. By convention, receivers listen on port 9997, but you can specify any unused port. You can use a tool like netstat to determine what ports are available on your system. Make sure the port you select is not in use by splunkweb or splunkd.

6. Click Save. You must restart Splunk to complete the process.

Set up receiving with Splunk CLI

To access the CLI, first navigate to $SPLUNK_HOME/bin/. This is unnecessary if you have added Splunk to your path.

To enable receiving, enter:

./splunk enable listen <port> -auth <username>:<password>

For <port>, substitute the port you want the receiver to listen on (the receiving port). For example, if you enter "9997," the receiver will receive data on port 9997. By convention, receivers listen on port 9997, but you can specify any unused port. You can use a tool like netstat to determine what ports are available on your system. Make sure the port you select is not in use by splunkweb or splunkd.

To disable receiving, enter:

./splunk disable listen -port <port> -auth <username>:<password>

Set up receiving with the configuration file

You can enable receiving on your Splunk instance by configuring inputs.conf in $SPLUNK_HOME/etc/system/local.

For most purposes, you just need to add a [splunktcp] stanza that specifies the receiving port. In this example, the receiving port is 9997:

[splunktcp://9997]

For further details, refer to the inputs.conf spec file.

To configure a universal forwarder as an intermediate forwarder (a forwarder that functions also as a receiver), use this method.

Searching data received from a forwarder running on a different operating system

In most cases, a Splunk instance receiving data from a forwarder on a different OS will need to install the app for that OS. However, there are numerous subtleties that affect this; read on for the details.

Forwarding and indexing are OS-independent operations. Splunk supports any combination of forwarders and receivers, as long as each is running on a certified OS. For example, a Linux receiver can index data from a Windows universal forwarder.

Once data has been forwarded and indexed, the next step is to search or perform other knowledge-based activities on the data. At this point, the Splunk instance performing such activities might need information about the OS whose data it is examining. You typically handle this by installing the app specific to that OS. For example, if you want a Linux Splunk instance to search OS-specific data forwarded from Windows, you will ordinarily want to install the Windows app on the Linux instance.

If the data you're interested in is not OS-specific, such as web logs, then you do not need to install the Splunk OS app.

In addition, if the receiver is only indexing the data, and an external search head is performing the actual searches, you do not need to install the OS app on the receiver, but you might need to install it on the search head. As an alternative, you can use a search head running the OS. For example, to search data forwarded from Windows to a Linux receiver, you can use a Windows search head pointing to the Linux indexer as a remote search peer. For more information on search heads, see "Set up distributed search".

Important: After you have downloaded the relevant OS app, remove its inputs.conf file before enabling the app, to ensure that its default inputs are not added to your indexer. For the Windows app, the location is: %SPLUNK_HOME%\etc\apps\windows\default\inputs.conf.

In summary, you only need to install the app for the forwarder's OS on the receiver (or search head) if it will be performing searches on the forwarded OS data.

Troubleshoot forwarder to receiver connectivity

Confusing the receiver's receiver and management ports

As part of setting up a forwarder, you specify the receiver's hostname/IP_address and port. The forwarder uses these to send data to the receiver. Be sure to specify the port that was designated as the receiving port at the time the receiver was configured. If you mistakenly specify the receiver's management port, the receiver will generate an error similar to this:

splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - SSL Error = error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - ACCEPT_RESULT=-1 VERIFY_RESULT=0
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - SSL Error for fd from HOST:localhost.localdomain, IP:127.0.0.1, PORT:53075
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - SSL Error = error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - ACCEPT_RESULT=-1 VERIFY_RESULT=0
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - SSL Error for fd from HOST:localhost.localdomain, IP:127.0.0.1, PORT:53076
splunkd.log:03-01-2010 13:35:28.653 ERROR TcpInputFd - SSL Error = error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
splunkd.log:03-01-2010 13:35:28.654 ERROR TcpInputFd - ACCEPT_RESULT=-1 VERIFY_RESULT=0
splunkd.log:03-01-2010 13:35:28.654 ERROR TcpInputFd - SSL Error for fd from HOST:localhost.localdomain, IP:127.0.0.1, PORT:53077
splunkd.log:03-01-2010 13:35:28.654 ERROR TcpInputFd - SSL Error = error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
splunkd.log:03-01-2010 13:35:28.654 ERROR TcpInputFd - ACCEPT_RESULT=-1 VERIFY_RESULT=0

Closed receiver socket

If a receiving indexer's queues become full, it will close the receiver socket, to prevent additional forwarders from connecting to it. If a forwarder with load-balancing enabled can no longer forward to that receiver, it will send its data to another indexer on its list. If the fowarder does not employ load-balancing, it will hold the data until the problem is resolved.

The receiver socket will reopen automatically when the queue gets unclogged.

Typically, a receiver gets behind on the dataflow because it can no longer write data due to a full disk or because it is itself attempting to forward data to another Splunk instance that is not accepting data.

The following warning message will appear in splunkd.log if the socket gets blocked:

Stopping all listening ports. Queues blocked for more than N seconds. 

This message will appear when the socket reopens:

Started listening on tcp ports. Queues unblocked. 

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has around configuring forwarding.

• Was this topic useful?
• Post a comment