Monitor files and directories with inputs.conf
You can use
inputs.conf to monitor files and directories with Splunk Enterprise.
Inputs.conf provides the most configuration options for setting up a file monitor input. For Splunk Cloud, use Splunk Web to configure file monitoring inputs instead.
To configure an input to Splunk Enterprise, add a stanza to inputs.conf in
$SPLUNK_HOME/etc/system/local/ or your own custom application directory in
$SPLUNK_HOME/etc/apps/. These locations are on the machine that runs Splunk Enterprise.
You can configure multiple settings in an input stanza. If you do not specify a value for a setting, Splunk Enterprise uses the default for that setting. You can find the defaults for settings in
For more information about configuration files, see About configuration files.
Configure monitoring of files with inputs.conf
- On the machine that runs Splunk Enterprise, open a shell or command prompt.
- Change to the
- (Optional) If
inputs.confdoes not exist, create the file.
inputs.conffor editing with a text editor.
- Add a stanza that references the files or directories that you want to monitor.
For example, to monitor the
/var/log/messagesfile on a *nix system, specify:
[monitor:///var/log/messages] disabled = 0
C:\Windows\System32\WindowsUpdate.logon a Windows system, specify:
[monitor://C:\Windows\System32\WindowsUpdate.log] disabled = 0
- (Optional) Add settings that further configure the input, depending on what you want the input to do. See Configuration settings later in this topic, or review the inputs.conf configuration specification file in the Admin Manual for additional settings.
[monitor://path/to/file] disabled = 0 setting1 = value setting2 = value ...
- Save the
inputs.conffile and close it.
- Either restart Splunk Enterprise or reload the configuration by running the following command. Splunk Enterprise prompts you for credentials if you reload the configuration.
./splunk _internal call /services/data/inputs/monitor/_reload -auth
You can use the following settings in both
batch input stanzas.
||Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. Splunk Enterprise prepends the
||the IP address or fully-qualified domain name of the host where the data originated.|
||Sets the index where events from this input will be stored. Splunk Enterprise prepends the
For more information about the index field, see How indexing works in the Managing Indexers and Clusters manual.
||Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to letting Splunk Enterprise determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
Sets the sourcetype key initial value. Splunk Enterprise uses the key during parsing and indexing to set the source type field and uses the source type field during searching. Splunk Enterprise prepends the
For more information about source types, see Why source types matter.
|Splunk Enterprise picks a source type based on various aspects of the data. There is no default.|
||Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply
||Specifies a comma-separated list of tcpout group names. Use this setting to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data.
Define the tcpout group names in
|the groups present in 'defaultGroup' in |
||A regular expression that extracts host from the file name of each input. Specifically, Splunk Enterprise uses the first group of the regular expression as the host.||the default "|
||Sets the segment of the path as the host, using
||the default "|
Monitor input stanzas configure Splunk Enterprise to watch all files in the
<path> or the
<path> itself if it represents a single file. You must specify the input type before the path, so add three slashes in the path if the path includes the root directory on *nix machines.
You can use wildcards for the path. See Specify input paths with wildcards.
[monitor://<path>] <setting1> = <val1> <setting2> = <val2> ...
The following are additional settings you can use when defining monitor input stanzas:
||Sets the source field for events from this input. You can use this setting when using the
Splunk Enterprise prepends the
|the input file path (except in the case of |
||Forces Splunk Enterprise to index files that have matching cyclic redundancy checks (CRCs). By default, the software only performs CRCs against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs.)
If set, Splunk Enterprise adds
Use caution with this setting for rolling log files. It can lead to the log file being re-indexed after it has rolled.
This setting is case sensitive.
||Causes the input to stop checking files for updates if the file modification time (modtime) has passed the
Splunk Enterprise does not index files whose modification time falls outside
You must specify
||If set to 1, monitoring begins at the end of the file (like *nix
||If set, Splunk Enterprise only monitors files whose names match the specified regular expression.||N/A|
||If set, Splunk Enterprise does NOT monitor files whose names match the specified regular expression.||N/A|
||If set to 1, Splunk Enterprise opens a file to check if it has already been indexed. This is only useful for files that don't update their modification time.
Use this setting for monitoring files on Windows, and for Internet Information Server (IIS) logs.
Caution: Use of this setting increases load and slows down indexing.
||If set to
||The modification time delta required before Splunk Enterprise can close a file on End-of-file (EOF). Tells the system not to close files that have been updated in the past
On Windows systems only, the
MonitorNoHandle input monitors files without using Windows file handles. This lets you read special log files like the Windows DNS server log files.
You must specify a valid path to a file when you use the
MonitorNoHandle input stanza. You cannot specify a directory. If you specify a file that already exists, Splunk Enterprise does not index the existing data in the file. It only indexes new data that the system writes to the file.
MonitorNoHandle sets the source for files you monitor to
MonitorNoHandle.To specify the file name as the source, you must use the
source setting in the stanza for the
MonitorNoHandle input for the file.
You can only configure
MonitorNoHandle on a Windows machine with inputs.conf or the Command Line Interface (CLI). You cannot configure it in Splunk Web or on a *nix machine.
[MonitorNoHandle://<path>] disabled=0 source = <path> <setting1> = <val1> <setting2> = <val2> ...
Use batch to set up a one-time, destructive input of data from a source.
For continuous, non-destructive inputs, use
monitor. Splunk enterprise deletes data that it has indexed with the
[batch://<path>] move_policy = sinkhole <setting1> = <val1> <setting2> = <val2> ...
When you define batch inputs, you must include the setting
move_policy = sinkhole. This loads the file destructively. Do not use the batch input type for files that you do not want to delete after indexing.
To ensure that Splunk Enterprise indexes new events when you copy over an existing file with new contents, set the
CHECK_METHOD = modtime setting in props.conf for the input source. This checks the modification time of the file and re-indexes it when the time changes. Splunk Enterprise indexes the entire file, which can result in duplicate events.
Examples of monitor input stanzas
Single *nix file
This example stanza configures Splunk Enterprise to index the single file,
[monitor:///var/log/messages] disabled = 0 sourcetype = unixlog
Single Windows directory
This Windows example configures Splunk Enterprise to monitor the directory,
C:\Windows\Logs. and all the files in it.
[monitor://C:\Windows\Logs] disabled = 0
Single Windows directory with spaces in filename
This Windows example configures Splunk Enterprise to monitor the directory,
C:\Program Files\VMWare. and all the files in it.
[monitor://C:\Program Files\VMWare] disabled = 0
Multiple Windows directories
This Windows example tells Splunk Enterprise to monitor all of the directories in
[monitor://C:\Windows\Debug\*] disabled = 0
Multiple *nix directories with a wildcard
This example configures Splunk Enterprise to monitor directories like
Multiple *nix files in one directory with a wildcard
This *nix example configures Splunk Enterprise to monitor multiple files in one directory, such as
MonitorNoHandle, single Windows file
This Windows example uses the MonitorNoHandle input to monitor a file that Windows has open for writing, such as
[MonitorNoHandle://<path>] disabled = 0 source = <path> <setting1> = <val1> <setting2> = <val2> ...
This batch example loads and deletes all files from the directory
[batch://system/flight815/*] move_policy = sinkhole
Monitor Splunk Enterprise files and directories with the CLI
Specify input paths with wildcards
This documentation applies to the following versions of Splunk® Enterprise: 6.5.7, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13
Feedback submitted, thanks!