Monitor files and directories with inputs.conf
You can set multiple attributes in an input stanza. If you do not specify a value for an attribute, the forwarder uses the default for that attribute, as defined in
For more information about configuration files, see About configuration files in the Splunk Enterprise Admin Manual.
Use the following attributes in both
batch input stanzas.
||Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. The forwarder prepends the
||the IP address or fully-qualified domain name of the host where the data originated.|
||Sets the index where events from this input will be stored. The forwarder prepends the
For more information about the index field, see How indexing works in the Splunk Enterprise Managing Indexers and Clusters manual.
||Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to letting the forwarder determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
Sets the sourcetype key initial value. The forwarder uses the key during parsing and indexing to set the source type field and uses the source type field during searching. The forwarder prepends the
For more information about source types, see Why source types matter.
|The forwarder picks a source type based on various aspects of the data. There is no default.|
||Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply
||Specifies a comma-separated list of tcpout group names. Use this attribute to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data.
Define the tcpout group names in
|the groups present in 'defaultGroup' in |
||A regular expression that extracts host from the file name of each input. Specifically, the forwarder uses the first group of the regular expression as the host.||the default "|
||Sets the segment of the path as the host, using
||the default "|
Monitor syntax and examples
Monitor input stanzas direct the forwarder to watch all files in the
<path> itself if it represents a single file). You must specify the input type and then the path, so put three slashes in the path if the path includes the root directory.
You can use wildcards for the path. See Specify input paths with wildcards.
[monitor://<path>] <attrbute1> = <val1> <attrbute2> = <val2> ...
The following are additional attributes you can use when defining monitor input stanzas:
|source = <string>||Sets the source field for events from this input. Do not override unless absolutely necessary, or when using the
The forwarder prepends the
|the input file path (except in the case of |
||Forces the forwarder to consume files that have matching CRCs (cyclic redundancy checks). By default, the software only performs CRC checks against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs, particularly if they have identical headers.)
If set, the forwarder adds
Use caution with this attribute for rolling log files. It can lead to the log file being re-indexed after it has rolled.
This setting is case sensitive.
||Causes the input to stop checking files for updates if their modification time (modtime) has passed the
The forwarder does not index files whose modification time falls outside
You must specify
||If set to 1, monitoring begins at the end of the file (like *nix
||If set, the forwarder only monitors files whose names match the specified regular expression.||N/A|
||If set, the forwarder does NOT monitor files whose names match the specified regular expression.||N/A|
||If set to 1, the forwarder opens a file to check if it has already been indexed. This is only useful for files that don't update their modification time.
Use this attribute for monitoring files on Windows, and mainly for Internet Information Server (IIS) logs.
Caution: Use of this attribute increases load and slows down indexing.
||If set to
||The modification time delta required before the forwarder can close a file on End-of-file (EOF). Tells the system not to close files that have been updated in the past
Example 1. To load anything in
Example 2. To load anything in
/apache/ that ends in
MonitorNoHandle syntax and examples
On Windows systems only, use the
MonitorNoHandle stanza to monitor files without using Windows file handles. This lets you read special log files like the Windows DNS server log files.
You must specify a valid path to a file when you use
MonitorNoHandle. You cannot specify a directory. If you specify a file that already exists, The forwarder does not index the existing data in the file. It only indexes new data that the system writes to the file.
MonitorNoHandle sets the source for files you monitor to
MonitorNoHandle. If you want to specify the file name as the source, you must specify it with the
source setting in the stanza for the
MonitorNoHandle input for the file.
You can only configure
MonitorNoHandle using inputs.conf or the CLI. you cannot configure it in Splunk Web.
[MonitorNoHandle://<path>] source = <path> <attrbute1> = <val1> <attrbute2> = <val2> ...
Batch syntax and examples
Use batch to set up a one time, destructive input of data from a source. For continuous, non-destructive inputs, use monitor. Remember, after the batch input is indexed, the forwarder deletes the file.
[batch://<path>] move_policy = sinkhole <attrbute1> = <val1> <attrbute2> = <val2> ...
When you define batch inputs, you must include the attribute,
move_policy = sinkhole. This loads the file destructively. Do not use the batch input type for files that you do not want to delete after indexing.
Example: This example batch loads all files from the directory
system/flight815/, but does not recurse through any subdirectories under it:
[batch://system/flight815/*] move_policy = sinkhole
Note: To ensure that new events are indexed when you copy over an existing file with new contents, set the
CHECK_METHOD = modtime attribute in props.conf for the source. See props.conf in the Splunk Enterprise Admin Manual for details. This checks the modification time of the file and re-indexes it when it changes. Be aware that the entire file will be re-indexed, which can result in duplicate events.
Monitor files and directories on a forwarder
Specify input paths with wildcards
This documentation applies to the following versions of Splunk Cloud™: 7.1.3, 7.1.6, 7.2.3, 7.2.4, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 8.0.0, 8.0.1