
Monitor files and directories with inputs.conf
To configure an input on your forwarder, add a stanza to inputs.conf in $SPLUNK_HOME/etc/system/local/
, or your own custom application directory in $SPLUNK_HOME/etc/apps/
.
You can set multiple attributes in an input stanza. If you do not specify a value for an attribute, the forwarder uses the default for that attribute, as defined in $SPLUNK_HOME/etc/system/default/inputs.conf
.
For more information about configuration files, see About configuration files in the Splunk Enterprise Admin Manual.
Configuration settings
Use the following attributes in both monitor
and batch
input stanzas.
Attribute | Description | Default |
---|---|---|
host = <string>
|
Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. The forwarder prepends the <string> with host:: .
|
the IP address or fully-qualified domain name of the host where the data originated. |
index = <string>
|
Sets the index where events from this input will be stored. The forwarder prepends the <string> with index:: .
For more information about the index field, see How indexing works in the Splunk Enterprise Managing Indexers and Clusters manual. |
main or whatever you set the default index to
|
sourcetype = <string>
|
Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to letting the forwarder determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
Sets the sourcetype key initial value. The forwarder uses the key during parsing and indexing to set the source type field and uses the source type field during searching. The forwarder prepends the For more information about source types, see Why source types matter. |
The forwarder picks a source type based on various aspects of the data. There is no default. |
queue = parsingQueue | indexQueue
|
Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.
|
parsingQueue
|
_TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>,...
|
Specifies a comma-separated list of tcpout group names. Use this attribute to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data.
Define the tcpout group names in |
the groups present in 'defaultGroup' in [tcpout] stanza in outputs.conf
|
host_regex = <regular expression>
|
A regular expression that extracts host from the file name of each input. Specifically, the forwarder uses the first group of the regular expression as the host. | the default "host = " attribute, if the regular expression fails to match
|
host_segment = <integer>
|
Sets the segment of the path as the host, using <integer> to determine the segment. For example, if host_segment = 2 , host becomes the second segment of the path. Path segments are separated by the '/' character.
|
the default "host = " attribute, if the value is not an integer, or is less than 1
|
Monitor syntax and examples
Monitor input stanzas direct the forwarder to watch all files in the <path>
(or <path>
itself if it represents a single file). You must specify the input type and then the path, so put three slashes in the path if the path includes the root directory.
You can use wildcards for the path. See Specify input paths with wildcards.
[monitor://<path>] <attrbute1> = <val1> <attrbute2> = <val2> ...
The following are additional attributes you can use when defining monitor input stanzas:
Attribute | Description | Default |
---|---|---|
source = <string> | Sets the source field for events from this input. Do not override unless absolutely necessary, or when using the MonitorNoHandle input and want to set the source to the name of the file you are monitoring. Consider use of source types, tagging, and search wildcards instead. The input layer usually provides a more accurate string to aid in problem analysis and investigation by accurately recording the file from which the data was retrieved.
The forwarder prepends the |
the input file path (except in the case of MonitorNoHandle , where it is MonitorNoHandle .)
|
crcSalt = <string>
|
Forces the forwarder to consume files that have matching CRCs (cyclic redundancy checks). By default, the software only performs CRC checks against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs, particularly if they have identical headers.)
If set, the forwarder adds Use caution with this attribute for rolling log files. It can lead to the log file being re-indexed after it has rolled. This setting is case sensitive. |
N/A |
ignoreOlderThan = <time_window>
|
Causes the input to stop checking files for updates if their modification time (modtime) has passed the <time_window> threshold. This improves the speed of file tracking operations when monitoring directory hierarchies with large numbers of historical files (for example, when active log files share a directory with old files that no longer get writes).
The forwarder does not index files whose modification time falls outside You must specify |
0 (disabled) |
followTail = 0|1
|
If set to 1, monitoring begins at the end of the file (like *nix tail -f ). This only applies to files the first time the forwarder attempts to monitor them. After that, the forwarder keeps track of the file using its internal file position records.
|
0 |
whitelist = <regular expression>
|
If set, the forwarder only monitors files whose names match the specified regular expression. | N/A |
blacklist = <regular expression>
|
If set, the forwarder does NOT monitor files whose names match the specified regular expression. | N/A |
alwaysOpenFile = 0 | 1
|
If set to 1, the forwarder opens a file to check if it has already been indexed. This is only useful for files that don't update their modification time.
Use this attribute for monitoring files on Windows, and mainly for Internet Information Server (IIS) logs. Caution: Use of this attribute increases load and slows down indexing. |
N/A |
recursive = true|false
|
If set to false , the forwarder does not look into subdirectories that it finds within a monitored directory.
|
true |
time_before_close = <integer>
|
The modification time delta required before the forwarder can close a file on End-of-file (EOF). Tells the system not to close files that have been updated in the past <integer> seconds.
|
3 |
followSymlink = true|false
|
If false , the forwarder ignores symbolic links that it finds within a monitored directory.
|
true |
Example 1. To load anything in /apache/foo/logs
or /apache/bar/logs
, etc.
[monitor:///apache/.../logs]
Example 2. To load anything in /apache/
that ends in .log
.
[monitor:///apache/*.log]
MonitorNoHandle syntax and examples
On Windows systems only, use the MonitorNoHandle
stanza to monitor files without using Windows file handles. This lets you read special log files like the Windows DNS server log files.
You must specify a valid path to a file when you use MonitorNoHandle
. You cannot specify a directory. If you specify a file that already exists, The forwarder does not index the existing data in the file. It only indexes new data that the system writes to the file.
MonitorNoHandle
sets the source for files you monitor to MonitorNoHandle
. If you want to specify the file name as the source, you must specify it with the source
setting in the stanza for the MonitorNoHandle
input for the file.
You can only configure MonitorNoHandle
using inputs.conf or the CLI. you cannot configure it in Splunk Web.
[MonitorNoHandle://<path>] source = <path> <attrbute1> = <val1> <attrbute2> = <val2> ...
Batch syntax and examples
Use batch to set up a one time, destructive input of data from a source. For continuous, non-destructive inputs, use monitor. Remember, after the batch input is indexed, the forwarder deletes the file.
[batch://<path>] move_policy = sinkhole <attrbute1> = <val1> <attrbute2> = <val2> ...
When you define batch inputs, you must include the attribute, move_policy = sinkhole
. This loads the file destructively. Do not use the batch input type for files that you do not want to delete after indexing.
Example: This example batch loads all files from the directory system/flight815/
, but does not recurse through any subdirectories under it:
[batch://system/flight815/*] move_policy = sinkhole
Note: To ensure that new events are indexed when you copy over an existing file with new contents, set the CHECK_METHOD = modtime
attribute in props.conf for the source. See props.conf in the Splunk Enterprise Admin Manual for details. This checks the modification time of the file and re-indexes it when it changes. Be aware that the entire file will be re-indexed, which can result in duplicate events.
PREVIOUS Monitor files and directories on a forwarder |
NEXT Specify input paths with wildcards |
This documentation applies to the following versions of Splunk Cloud™: 7.2.4, 7.2.6, 7.2.7, 7.2.8, 7.2.9
Feedback submitted, thanks!