
Get data from TCP and UDP ports
You can enable Splunk to accept an input on any TCP or UDP port. Splunk consumes any data sent on these ports. Use this method for syslog (default port is UDP 514) or set up netcat and bind to a port.
TCP is the protocol underlying Splunk's data distribution and is the recommended method for sending data from any remote machine to your Splunk server. Splunk can index remote data from syslog-ng or any other application that transmits via TCP.
Splunk supports monitoring over UDP, but recommends using TCP instead whenever possible. UDP is generally undesirable as a transport because:
- It doesn't enforce delivery.
- It's not encrypted.
- There's no accounting for lost datagrams.
Refer to "Working with UDP connections" on the Splunk Community Wiki for recommendations if you must use UDP.
Add a network input using Splunk Web
To add inputs from network ports using Splunk Web:
A. Go to the Add New page
You add a network input from the Add New page in Splunk Web. You can get there through two routes:
- Splunk Manager
- Splunk Home
It doesn't matter which route you use to get there; the Add New page itself is the same either way.
Via Splunk Manager:
1. Click Manager in the upper right-hand corner of Splunk Web.
2. In the Data section of the Manager page, click Data Inputs.
3. Pick TCP or UDP.
4. Click the New button to add an input.
Via Splunk Home:
1. Click the Add Data link in Splunk Home. This brings you to a page called "Data recipes".
2. Click either the From a TCP port or the From a UDP port link to add an input.
B. Specify the network input
1. Enter a port number. The user you run Splunk as must have access to the port. On a Unix system, you must run as root to access a port under 1024.
2. If this is a TCP input, you can specify whether this port should accept connections from all hosts or one host. If you specify one host, enter the IP address of the host.
3. Enter a new Source name to override the default source value, if necessary.
Important: Consult Splunk support before changing this value.
4. To access other settings, check More settings. A number of additional settings appear. You can usually go with the defaults for these settings. If you want to set them explicitly, here's what they're for:
a. You can set the Host by selecting a radio button:
- IP. Sets the input processor to rewrite the host with the IP address of the remote server.
- DNS. Sets the host to the DNS entry of the remote server.
- Custom. Sets the host to a user-defined label.
b. You can set the Source type. Source type is a default field added to events. Source type is used to determine processing characteristics, such as timestamps and event boundaries. For information on overriding Splunk's automatic source typing, see "Override automatic source type assignment" in this manual.
c. You can set the Index. Leave the value as "default" unless you have defined multiple indexes to handle different types of events. In addition to indexes for user data, Splunk has a number of utility indexes, which also appear in this dropdown box.
5. Click Save.
Add a network input using the CLI
To access Splunk's CLI, navigate to the $SPLUNK_HOME/bin/
directory and use the ./splunk
command.
If you get stuck, Splunk's CLI has built-in help. Access the main CLI help by typing splunk help
. Individual commands have their own help pages as well; type splunk help <command>
.
The following CLI commands are available for network input configuration:
Command | Command syntax | Action |
---|---|---|
add | add tcp | udp $SOURCE [-parameter value] ...
|
Add inputs from $SOURCE .
|
edit | edit tcp | udp $SOURCE [-parameter value] ...
|
Edit a previously added input for $SOURCE .
|
remove | remove tcp | udp $SOURCE
|
Remove a previously added data input. |
list | list tcp | udp
|
List the currently configured monitor. |
Change the configuration of each data input type by setting additional parameters. Parameters are set with this syntax:
-<parameter> <value>
Parameter | Required? | Description |
---|---|---|
$SOURCE
|
Yes | Port number to listen for data to index. The user you run Splunk as must have access to this port. On a Unix system, you must run as root to access a port under 1024. |
sourcetype
|
No | Specify a sourcetype field value for events from the input source. |
index
|
No | Specify the destination index for events from the input source. |
hostname
|
No | Specify a host name to set as the host field value for events from the input source. |
remotehost
|
No | Specify an IP address to exclusively accept data from. |
resolvehost
|
No | Set True or False (T | F). Default is False. Set True to use DNS to set the host field value for events from the input source. |
Examples
- Configure a UDP input to watch port 514 and set the source type to "syslog":
./splunk add udp 514 -sourcetype syslog
- Set the UDP input's host value via DNS. Use
auth
with your username and password:
./splunk edit udp 514 -resolvehost true -auth admin:changeme
Check the Best Practices Wiki for information about the best practices for using UDP when configuring Syslog input.
Add a network input using inputs.conf
To add an input, add a stanza for it to inputs.conf in $SPLUNK_HOME/etc/system/local/
, or your own custom application directory in $SPLUNK_HOME/etc/apps/
. If you have not worked with Splunk's configuration files before, read "About configuration files" in the Admin manual before you begin.
You can set any number of attributes and values following an input type. If you do not specify a value for one or more attributes, Splunk uses the defaults that are preset in $SPLUNK_HOME/etc/system/default/
(noted below).
TCP
[tcp://<remote server>:<port>] <attrbute1> = <val1> <attrbute2> = <val2> ...
This type of input stanza tells Splunk to listen to <remote server> on <port>. If <remote server>
is blank, Splunk listens to all connections on the specified port.
Note: The user you run Splunk as must have access to the listening port. On a Unix system, you must run as root to access a port under 1024.
host = <string>
- Sets the host key/field to a static value for this stanza.
- Sets the host key's initial value. The key is used during parsing/indexing, in particular to set the host field. It is also the host field used at search time.
- The
<string>
is prepended with 'host::'. - If not set explicitly, this defaults to the IP address or fully qualified domain name of the host where the data originated.
index = <string>
- Set the index where events from this input will be stored.
- The
<string>
is prepended with 'index::'. - Defaults to
main
, or whatever you have set as your default index. - For more information about the index field, see "How indexing works" in the Admin manual.
sourcetype = <string>
- Sets the sourcetype key/field for events from this input.
- Explicitly declares the source type for this data, as opposed to allowing it to be determined automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
- Sets the sourcetype key's initial value. The key is used during parsing/indexing, in particular to set the source type field during indexing. It is also the source type field used at search time.
- The
<string>
is prepended with 'sourcetype::'. - If not set explicitly, Splunk picks a source type based on various aspects of the data. There is no hard-coded default.
- For more information about source types, see "Why source types matter", in this manual.
source = <string>
- Sets the source key/field for events from this input.
- Note: Overriding the source key is generally not recommended. Typically, the input layer will provide a more accurate string to aid in problem analysis and investigation, accurately recording the file from which the data was retreived. Consider use of source types, tagging, and search wildcards before overriding this value.
- The
<string>
is prepended with 'source::'. - Defaults to the input file path.
queue = [parsingQueue|indexQueue]
- Specifies where the input processor should deposit the events that it reads.
- Set to "parsingQueue" to apply
props.conf
and other parsing rules to your data. - Set to "indexQueue" to send your data directly into the index.
- Defaults to
parsingQueue
.
connection_host = [ip|dns|none]
- "ip" sets the host to the IP address of the remote server.
- "dns" sets the host to the DNS entry of the remote server.
- "none" leaves the host as specified.
- Defaults to "ip".
TCP over SSL
[tcp-ssl:<port>]
Use this stanza type if you are receiving encrypted, unparsed data from a forwarder or third-party system. Set <port>
to the port on which the forwarder or third-party system is sending unparsed, encrypted data.
UDP
[udp://<remote server>:<port>] <attrbute1> = <val1> <attrbute2> = <val2> ...
This type of input stanza is similar to the TCP type, except that it listens on a UDP port.
Note:
- If
<remote server>
is specified, the specified port will only accept data from that server. - If
<remote server>
is empty -[udp://<port>]
- the port will accept data sent from any server.
host = <string>
- Sets the host key/field to a static value for this stanza.
- Sets the host key's initial value. The key is used during parsing/indexing, in particular to set the host field. It is also the host field used at search time.
- The
<string>
is prepended with 'host::'. - If not set explicitly, this defaults to the IP address or fully qualified domain name of the host where the data originated.
index = <string>
- Set the index where events from this input will be stored.
- The
<string>
is prepended with 'index::'. - Defaults to
main
, or whatever you have set as your default index. - For more information about the index field, see "How indexing works" in the Admin manual.
sourcetype = <string>
- Sets the sourcetype key/field for events from this input.
- Explicitly declares the source type for this data, as opposed to allowing it to be determined automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
- Sets the sourcetype key's initial value. The key is used during parsing/indexing, in particular to set the source type field during indexing. It is also the source type field used at search time.
- The
<string>
is prepended with 'sourcetype::'. - If not set explicitly, Splunk picks a source type based on various aspects of the data. There is no hard-coded default.
- For more information about source types, see "Why source types matter", in this manual.
source = <string>
- Sets the source key/field for events from this input.
- Note: Overriding the source key is generally not recommended. Typically, the input layer will provide a more accurate string to aid in problem analysis and investigation, accurately recording the file from which the data was retreived. Consider use of source types, tagging, and search wildcards before overriding this value.
- The
<string>
is prepended with 'source::'. - Defaults to the input file path.
queue = [parsingQueue|indexQueue]
- Specifies where the input processor should deposit the events that it reads.
- Set to "parsingQueue" to apply
props.conf
and other parsing rules to your data. - Set to "indexQueue" to send your data directly into the index.
- Defaults to
parsingQueue
.
_rcvbuf = <integer>
- Specify the receive buffer for the UDP port, measured in bytes.
- If the value is 0 or negative, it is ignored.
- Defaults to 1,572,864.
- Note: If the default value is too large for an OS, Splunk will try to set the value to 1572864/2. If that value also fails, Splunk will retry with 1572864/(2*2). It will continue to retry by halving the value until it succeeds.
no_priority_stripping = [true|false]
- Setting for receiving syslog data.
- If this attribute is set to true, Splunk does NOT strip the <priority> syslog field from received events.
- Depending on how this attribute is set, Splunk sets event timestamps differently:
- When set to true, Splunk honors the timestamp as it comes from the source.
- When set to false, Splunk assigns events the local time.
- Default is false (Splunk strips <priority>).
no_appending_timestamp = [true|false]
- If this attribute is set to true, Splunk does NOT append a timestamp and host to received events.
- Note: Do NOT include this attribute if you want to append timestamp and host to received events.
- Default is false.
UDP packets and line merging
When using UDP as an input, there is an implicit expectation that each UDP packet will be indexed as an independent event. However, this is not true. Splunk performs event merging on the datastream, so events will be merged together if they don't have a clear timestamp.
In the case of single-line events, you can avoid this problem by editing the underlying source type in props.conf
and setting the SHOULD_LINEMERGE
attribute to false
. By doing so, you will keep Splunk from merging packets together.
Answers
Have questions? Visit Splunk Answers and see what and answers the Splunk community has about questions UDP inputs, TCP inputs, and inputs in general,
PREVIOUS How Splunk handles log file rotation |
NEXT Send SNMP events to Splunk Enterprise |
This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7
I'd love to see some example configuration settings for this topic.