Getting Data In

Get data from TCP and UDP ports

You can configure Splunk Enterprise to accept an input on any TCP or UDP port. Splunk Enterprise consumes any data sent on these ports. Use this method to capture data from network services such as syslog (default port is UDP 514). You can also set up the netcat service and bind it to a port.

TCP is the network protocol that underlies the Splunk Enterprise data distribution and is the recommended method for sending data from any remote machine to your Splunk Enterprise server. Splunk Enterprise can index remote data from syslog-ng or any other application that transmits via TCP.

Splunk Enterprise supports monitoring over UDP, but recommends using TCP instead whenever possible. UDP is generally undesirable as a transport because, among other reasons, it doesn't guarantee delivery.

Refer to "Working with UDP connections" on the Splunk Community Wiki for recommendations if you must use UDP.

Add a network input using Splunk Web

To add inputs from network ports using Splunk Web:

A. Go to the Add New page

You add a network input from the Add New page in Splunk Web. You can get there through two routes:

• Splunk Settings
• Splunk Home

It doesn't matter which route you use to get there; the Add New page itself is the same either way.

Via Splunk Settings:

1. Click Settings in the upper right-hand corner of Splunk Web.

2. In the Data section of the Settings pop-up, click Data Inputs.

3. Pick TCP or UDP.

4. Click the New button to add an input.

Via Splunk Home:

1. Click the Add Data link in Splunk Home. This brings you to a page called "Data recipes".

2. Click either the From a TCP port or the From a UDP port link to add an input.

B. Specify the network input

1. Enter a port number. The user you run Splunk Enterprise as must have access to the port. On a stock Unix system, you must run Splunk as root to listen on a port under 1024.

2. If this is a TCP input, you can specify whether this port should accept connections from all hosts or one host.

• If you only want the input to accept connections from one host, select the No, restrict to one host radio button under Accept connections from all hosts?
• In the Host restriction field that appears, enter the host name or IP address of the host.

3. Enter a new Source name to override the default source value, if necessary.

Important: Consult Splunk Support before changing this value.

4. To access other settings, check More settings. A number of additional settings appear. You can usually go with the defaults for these settings. If you want to set them explicitly, here's what they're for:

a. You can set the Host by selecting a radio button:

• IP. Sets the input processor to rewrite the host with the IP address of the remote server.

• DNS. Sets the host to the DNS entry of the remote server.

• Custom. Sets the host to a user-defined label.

b. You can set the Source type. Source type is a default field added to events. Source type is used to determine processing characteristics, such as timestamps and event boundaries. For information on overriding Splunk's automatic source typing, see "Override automatic source type assignment" in this manual.

c. You can set the Index. Leave the value as "default" unless you have defined multiple indexes to handle different types of events. In addition to indexes for user data, Splunk has a number of utility indexes, which also appear in this dropdown box.

5. Click Save.

Add a network input using the CLI

To access the Splunk Enterprise CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command.

If you get stuck, the CLI has built-in help. Access the main CLI help by typing splunk help. Individual commands have their own help pages as well; type splunk help <command>.

The following CLI commands are available for network input configuration:

Command	Command syntax	Action
add	add tcp|udp <port> [-parameter value] ...	Add inputs from <port>.
edit	edit tcp|udp <port> [-parameter value] ...	Edit a previously added input for <port>.
remove	remove tcp|udp <port>	Remove a previously added data input.
list	list tcp|udp [<port>]	List the currently configured monitor.

The <port> is the port number on which to listen for data. The user you run Splunk as must have access to this port.

You can modify the configuration of each input by setting any of these additional parameters:

Parameter	Required?	Description
sourcetype	No	Specify a sourcetype field value for events from the input source.
index	No	Specify the destination index for events from the input source.
hostname	No	Specify a host name to set as the host field value for events from the input source.
remotehost	No	Specify an IP address to exclusively accept data from.
resolvehost	No	Set True of False (T | F). Default is False. Set True to use DNS to set the host field value for events from the input source.
restrictToHost	No	Specify a host name or IP address that this input should accept connections from only.

Examples

• Configure a UDP input to watch port 514 and set the source type to "syslog":

     ./splunk add udp 514 -sourcetype syslog

• Set the UDP input's host value via DNS. Use auth with your username and password:

     ./splunk edit udp 514 -resolvehost true -auth admin:changeme

Check the Best Practices Wiki for information about the best practices for using UDP when configuring Syslog input.

Change restricted hosts on a TCP network input

If, when creating a TCP input, you decide to only accept connections from a specific host, once you save that input, you can neither change nor remove that host later, either from Splunk Web or the CLI.

To change or remove the restricted host of a port, you must first delete the input that contains the old restricted host. Then, you must add a new input that either contains the new restricted host, or has no restriction.

Add a network input using inputs.conf

To add an input, add a stanza for it to inputs.conf in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. If you have not worked with Splunk's configuration files before, read "About configuration files" in the Admin manual before you begin.

You can set any number of attributes and values following an input type. If you do not specify a value for one or more attributes, Splunk uses the defaults that are preset in $SPLUNK_HOME/etc/system/default/ (noted below).

TCP

[tcp://<remote server>:<port>]
<attrbute1> = <val1>
<attrbute2> = <val2>
...

This type of input stanza tells Splunk Enterprise to listen to <remote server> on <port>. If <remote server> is blank, Splunk listens to all connections on the specified port.

Note: The user you run Splunk Enterprise as must have access to the listening port. On a *nix system, you must run as root to access a port under 1024.

Attribute	Description	Default
host = <string>	Sets the host key/field to a static value for this stanza. Sets the host key's initial value. The key is used during parsing/indexing, in particular to set the host field. It is also the host field used at search time. The <string> is prepended with 'host::'.	the IP address or fully-qualified domain name of the host where the data originated.
index = <string>	Set the index where events from this input will be stored. The <string> is prepended with 'index::'. For more information about the index field, see "How indexing works" in the Managing Indexers and Clusters manual.	main or whatever you set the default index to
sourcetype = <string>	Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to allowing Splunk Enterprise to determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing. Sets the sourcetype key's initial value. Splunk Enterprise uses the key during parsing/indexing, in particular to set the source type field during indexing. It is also the source type field used at search time. The <string> is prepended with 'sourcetype::'. For more information about source types, see "Why source types matter", in this manual.	Splunk Enterprise picks a source type based on various aspects of the data. There is no hard-coded default.
source = <string>	Sets the source key/field for events from this input. Note: Overriding the source key is generally not recommended. Typically, the input layer will provide a more accurate string to aid in problem analysis and investigation, accurately recording the file from which the data was retreived. Consider use of source types, tagging, and search wildcards before overriding this value. The <string> is prepended with 'source::'.	The input file path
queue = parsingQueue | indexQueue	Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.	parsingQueue
connection_host = ip | dns | none	"ip" sets the host to the IP address of the remote server. "dns" sets the host to the DNS entry of the remote server. "none" leaves the host as specified.	ip

TCP over SSL

[tcp-ssl:<port>]

Use this stanza type if you are receiving encrypted, unparsed data from a forwarder or third-party system. Set <port> to the port on which the forwarder or third-party system is sending unparsed, encrypted data.

UDP

[udp://<remote server>:<port>]
<attrbute1> = <val1>
<attrbute2> = <val2>
...

This type of input stanza is similar to the TCP type, except that it listens on a UDP port.

Note:

• If <remote server> is specified, the specified port will only accept data from that server.
• If <remote server> is empty - [udp://<port>] - the port will accept data sent from any server.

Attribute	Description	Default
host = <string>	Sets the host key/field to a static value for this stanza. Sets the host key's initial value. The key is used during parsing/indexing, in particular to set the host field. It is also the host field used at search time. The <string> is prepended with 'host::'.	the IP address or fully-qualified domain name of the host where the data originated.
index = <string>	Set the index where events from this input will be stored. The <string> is prepended with 'index::'. For more information about the index field, see "How indexing works" in the Managing Indexers and Clusters manual.	main or whatever you set the default index to
sourcetype = <string>	Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to allowing Splunk Enterprise to determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing. Sets the sourcetype key's initial value. Splunk Enterprise uses the key during parsing/indexing, in particular to set the source type field during indexing. It is also the source type field used at search time. The <string> is prepended with 'sourcetype::'. For more information about source types, see "Why source types matter", in this manual.	Splunk Enterprise picks a source type based on various aspects of the data. There is no hard-coded default.
source = <string>	Sets the source key/field for events from this input. Note: Overriding the source key is generally not recommended. Typically, the input layer will provide a more accurate string to aid in problem analysis and investigation, accurately recording the file from which the data was retreived. Consider use of source types, tagging, and search wildcards before overriding this value. The <string> is prepended with 'source::'.	The input file path
queue = parsingQueue | indexQueue	Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.	parsingQueue
_rcvbuf = <integer>	Specify the receive buffer for the UDP port, in bytes. If the value is 0 or negative, Splunk Enterprise ignores the value.	1,572,864 - however, if the default value is too large for an OS, Splunk Enterprise halves the value from this default continuously until the buffer size is at an acceptable level.
no_priority_stripping = true | false	Setting for receiving syslog data. If you set this attribute to true, Splunk Enterprise does NOT strip the <priority> syslog field from received events. Depending on how this attribute is set, Splunk Enterprise sets event timestamps differently: When set to true, Splunk Enterprise honors the timestamp as it comes from the source. When set to false, Splunk Enterprise assigns events the local time.	false (Splunk Enterprise strips <priority>.)
no_appending_timestamp = true | false	If you set this attribute to true, Splunk Enterprise does NOT append a timestamp and host to received events. Note: Do NOT include this attribute if you want to append timestamp and host to received events.	false

UDP packets and line merging

You might expect that Splunk Enterprise indexes each UDP packet as an independent event. However, this is not the case. Splunk Enterprise performs event merging on the datastream, and merges events together if they don't have a clear timestamp.

In the case of single-line events, you can avoid this problem by editing the underlying source type in props.conf and setting the SHOULD_LINEMERGE attribute to false. Doing so keeps Splunk Enterprise from merging packets together.

Answers

Have questions? Visit Splunk Answers and see what and answers the Splunk community has about questions UDP inputs, TCP inputs, and inputs in general,