Splunk® Enterprise

Getting Data In

Download manual as PDF

Download topic as PDF

Get data from TCP and UDP ports

You can configure Splunk Enterprise to accept an input on any TCP or UDP port. Splunk Enterprise consumes any data that arrives on these ports. Use this method to capture data from network services such as syslog (default port is UDP 514). You can also set up the netcat service and bind it to a port.

For security, Splunk Cloud accepts connections only from forwarders with the correct Secure Sockets Layer (SSL) certificates. If you want to send data from a TCP or UDP source such as syslog, use the Splunk Universal Forwarder to listen to the source and forward the data to your Splunk Cloud deployment.

TCP is the network protocol that underlies the Splunk Enterprise data distribution scheme. It is the recommended protocol for sending data from any remote host to your Splunk Enterprise server. Splunk Enterprise can index remote data from syslog-ng or any other application that transmits via TCP.

Splunk Enterprise supports monitoring over UDP, but you should use TCP to send network data instead whenever possible. UDP is not desirable as a transport because, among other reasons, it does not guarantee delivery of network packets.

When you monitor TCP network ports, the user Splunk Enterprise runs as must have access to the port you want to monitor. On many Unix operating systems, by default, you must run Splunk Enterprise as the root user to listen directly on a port below 1024.

See Working with UDP connections on the Splunk Community Wiki for recommendations if you must send network data with UDP.

Confirm how your network device handles external monitoring before you use the network monitoring input

Before you begin monitoring the output of a network device with the Splunk Enterprise network monitor, confirm how the device interacts with external network monitors.

If you configure TCP logging on some network devices, such as a Cisco Adaptive Security Appliance (ASA), and the device cannot connect to the monitor, it might cause reduced performance or stop logging, or worse. By default, the Cisco ASA stops accepting incoming network connections when it encounters network congestion or connectivity problems.

Add a network input using Splunk Web

To add inputs from network ports using Splunk Web:

Go to the Add New page

You can get there through two routes.

By Splunk Settings:

1. Click Settings.

2. Click Data Inputs.

3. Choose TCP or UDP.

4. Click New to add an input.

By Splunk Home:

1. Click the Add Data link in Splunk Home.

2. Click Monitor to monitor a network port on the local machine, or Forward to receive network data from another machine.

Note: Forwarding a file requires additional setup.

3. If you selected Forward, choose or create the group of forwarders you want this input to apply to.

4. Click Next.

Specify the network input

1. In the left pane, click TCP / UDP to add an input.

2. Click the TCP or UDP button to choose between a TCP or UDP input.

3. In the Port field, enter a port number.

4. In the Source name override field, enter a new source name to override the default source value, if necessary.

Note: Consult Splunk Support before changing the "Source name override" value.

5. If this is a TCP input, specify whether this port should accept connections from all hosts or only one host in the Only accept connections from field. If you only want the input to accept connections from one host, enter the host name or IP address of the host. You can use wildcards to specify hosts.

6. Click Next to continue to the Input Settings page.

Specify input settings

The Input Settings page lets you specify source type, application context, default host value, and index. All of these parameters are optional.

1. Set the Source type. This is a default field that Splunk Enterprise adds to events and uses to determine processing characteristics, such as timestamps and event boundaries.

2. Set the Host name value. You have several choices:

  • IP. Sets the input processor to rewrite the host with the IP address of the remote server.
  • DNS. Sets the host to the DNS entry of the remote server.
  • Custom. Sets the host to a user-defined label.

Learn more about setting the host value in "About hosts".

Note: Host only sets the host field in the resulting events. It does not direct Splunk Enterprise to look on a specific host on your network.

3. Set the Index that Splunk Enterprise should send data to for this input. Leave the value as "default" unless you have defined multiple indexes to handle different types of events. In addition to indexes for user data, Splunk Enterprise has a number of utility indexes, which also appear in this dropdown box.

4. Click Review.

Review your choices

After specifying all your input settings, review your selections. Splunk Enterprise lists the options you selected, including the type of monitor, the source, the source type, the application context, and the index.

1. Review the settings.

2. If they are not what you want, click < to go back to the previous step in the wizard. Otherwise, click Submit.

Splunk Enterprise then loads the "Success" page and begins indexing the specified network input.

Add a network input using the CLI

To access the Splunk Enterprise CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command.

If you get stuck, the CLI has help. Access the main CLI help by typing splunk help. Individual commands have their own help pages as well and can be accessed by typing splunk help <command>.

The following CLI commands are available for network input configuration:

Command Command syntax Action
add add tcp|udp <port> [-parameter value] ... Add inputs from <port>.
edit edit tcp|udp <port> [-parameter value] ... Edit a previously added input for <port>.
remove remove tcp|udp <port> Remove a previously added data input.
list list tcp|udp [<port>] List the currently configured monitor.

The <port> is the port number on which to listen for data. The user you run Splunk as must have access to this port.

You can modify the configuration of each input by setting any of these additional parameters:

Parameter Required? Description
sourcetype No Specify a sourcetype field value for events from the input source.
index No Specify the destination index for events from the input source.
hostname No Specify a host name to set as the host field value for events from the input source.
remotehost No Specify an IP address to exclusively accept data from.
resolvehost No Set to true or false (T | F). Default is False. Set to true to use DNS to set the host field value for events from the input source.
restrictToHost No Specify a host name or IP address that this input should accept connections from only.

Examples

  • Configure a UDP input to watch port 514 and set the source type to "syslog":
     ./splunk add udp 514 -sourcetype syslog
  • Set the UDP input host value via DNS. Use auth with your username and password:
     ./splunk edit udp 514 -resolvehost true -auth admin:changeme

For information on best practices for using UDP, see Best practices for configuring Syslog input in the Community Wiki.

Change restricted hosts on a TCP network input

If you decide to only accept connections from a specific host when you create a TCP input, once you save that input, you can neither change nor remove that host later, either from Splunk Web or the CLI.

To change or remove the restricted host of a port, you must first delete the input that contains the old restricted host. Then, you must add a new input that either contains the new restricted host or has no restriction.

Add a network input using inputs.conf

To add an input, add a stanza for it to inputs.conf in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. If you have not worked with Splunk's configuration files before, read About configuration files in the Admin manual before you begin.

You can set any number of attributes and values following an input type. If you do not specify a value for one or more attributes, Splunk Enterprise uses the defaults that are preset in $SPLUNK_HOME/etc/system/default/ (noted below).

Configure a TCP input

[tcp://<remote server>:<port>]
<attrbute1> = <val1>
<attrbute2> = <val2>
...

Tells Splunk Enterprise to listen to <remote server> on <port>. If <remote server> is blank, Splunk Enterprise listens to all connections on the specified port.

Attribute Description Default
host = <string> Sets the host key/field to a static value for this stanza. Also sets the host key initial value. Splunk Enterprise uses the key during parsing and indexing, in particular to set the host field. It also uses the host field at search time.

The <string> is prepended with 'host::'.

The IP address or fully-qualified domain name of the host where the data originated.
index = <string> Sets the index where Splunk Enterprise should store the events from this input. The <string> is prepended with 'index::'. main or whatever you set the default index to
sourcetype = <string> Sets the sourcetype key/field for events from this input. Also declares the source type for this data, instead of letting Splunk Enterprise determine it. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.

Sets the sourcetype key initial value. Splunk Enterprise uses the key during parsing and indexing, in particular to set the source type field during indexing. Splunk Enterprise uses the source type field used search time.

The <string> is prepended with 'sourcetype::'.

Splunk Enterprise picks a source type based on various aspects of the data. There is no hard-coded default.
source = <string> Sets the source key/field for events from this input. The <string> is prepended with 'source::'.

Note: Do not override the source key unless absolutely necessary. The input layer provides a more accurate string to aid in problem analysis and investigation by recording the file from which the data was retrieved. Consider use of source types, tagging, and search wildcards before overriding this value.

The input file path
queue = parsingQueue | indexQueue Specifies where the input processor should deposit the events that it reads.

Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.

parsingQueue
connection_host = ip | dns | none "ip" sets the host to the IP address of the remote server.

"dns" sets the host to the DNS entry of the remote server.

"none" leaves the host as specified.

ip

Configure a TCP input over SSL

[tcp-ssl:<port>]

Use this stanza type if you receive encrypted, unparsed data from a forwarder or third-party system. Set <port> to the port on which the forwarder or third-party system is sending unparsed, encrypted data.

Configure a UDP input

[udp://<remote server>:<port>]
<attrbute1> = <val1>
<attrbute2> = <val2>
...

This type of input stanza is similar to the TCP type, except that it listens on a UDP port.

  • If you specify <remote server>, the specified port only accepts data from that host.
  • If you specify nothing for <remote server> - [udp://<port>] - the port accepts data sent from any host.
Attribute Description Default
host = <string> Sets the host key/field to a static value for this stanza. Also sets the host key initial value. Splunk Enterprise uses this key during parsing and indexing, in particular to set the host field. It also uses the host field at search time. The <string> is prepended with 'host::'. The IP address or fully-qualified domain name of the host where the data originated.
index = <string> Sets the index where Splunk Enterprise should store events from this input. The <string> is prepended with 'index::'. main or whatever you set the default index to
sourcetype = <string> Sets the sourcetype key/field for events from this input. Also declares the source type for this data, as opposed to letting Splunk Enterprise determine it. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.

Sets the sourcetype key initial value. Splunk Enterprise uses the key during parsing and indexing, in particular to set the source type field during indexing. It also uses the source type field used search time.

The <string> is prepended with 'sourcetype::'.

Splunk Enterprise picks a source type based on various aspects of the data. There is no hard-coded default.
source = <string> Sets the source key/field for events from this input. The <string> is prepended with 'source::'.

Note: Do not override the source key unless absolutely necessary. The input layer provides a more accurate string to aid in problem analysis and investigation by recording the file from which the data was retrieved. Consider use of source types, tagging, and search wildcards before overriding this value.

The input file path
queue = parsingQueue | indexQueue Sets where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index. parsingQueue
_rcvbuf = <integer> Sets the receive buffer for the UDP port, in bytes. If the value is 0 or negative, Splunk Enterprise ignores the value. 1,572,864 unless the value is too large for an OS. In this case, Splunk Enterprise halves the value from this default continuously until the buffer size is at an acceptable level.
no_priority_stripping = true | false Sets how Splunk Enterprise handles receiving syslog data.

If you set this attribute to true, Splunk Enterprise does not strip the <priority> syslog field from received events.

Depending on how you set this attribute, Splunk Enterprise also sets event timestamps differently. When set to true, Splunk Enterprise honors the timestamp as it comes from the source. When set to false, Splunk Enterprise assigns events the local time.

false (Splunk Enterprise strips <priority>.)
no_appending_timestamp = true | false Sets how Splunk Enterprise applies timestamps and hosts to events.

If you set this attribute to true, Splunk Enterprise does not append a timestamp and host to received events.

Note: Do not set this attribute if you want to append timestamp and host to received events.

false (Splunk Enterprise appends timestamps and hosts to events)

UDP packets and line merging

Splunk Enterprise does not index each UDP packet as an independent event. Instead, it performs event merging on the data stream and merges events together if they don't have a clear timestamp.

You can avoid this problem by editing the underlying source type in props.conf and setting the SHOULD_LINEMERGE attribute to false. This keeps Splunk Enterprise from merging packets together.

Answers

Have questions? Visit Splunk Answers and see what and answers the Splunk community has about questions UDP inputs, TCP inputs, and inputs in general,

PREVIOUS
How Splunk Enterprise handles log file rotation
  NEXT
Set up and use HTTP Event Collector

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 7.0.0


Comments

Hi Andrey2007,

No, there is no support for wildcards with these stanzas. When you specify a remote server address, the instance only accepts IP traffic from that address.

Malmoore, Splunker
January 11, 2016

Does tcp or udp input support mask in ip address or hostname specification? Like [tcp://192.168.2.*:514]

Andrey2007
March 18, 2015

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters