Monitor files and directories with inputs.conf
You can use the inputs.conf file to monitor files and directories with the Splunk platform. The inputs.conf file provides the most configuration options for setting up a file monitor input. If you use Splunk Cloud, you can use either Splunk Web or a forwarder to configure file monitoring inputs.
To configure an input, add a stanza to the inputs.conf file in the $SPLUNK_HOME/etc/system/local/ directory or your own custom application directory in $SPLUNK_HOME/etc/apps/. These locations are on the machine that runs Splunk Enterprise or the forwarder. To learn more about the inputs.conf file, see inputs.conf in the Splunk Enterprise Admin Manual.
You can configure multiple settings in an input stanza. If you don't specify a value for a setting, the Splunk platform uses the default for that setting. You can find the defaults for settings in the $SPLUNK_HOME/etc/system/default/inputs.conf directory.
For more information about configuration files, see About configuration files in the Splunk Enterprise Admin Manual.
Configure a forwarder to send data to Splunk Cloud
If you want to send Active Directory (AD) data to Splunk Cloud, you must install and configure a forwarder before you begin making edits to configuration files on the forwarder.
- Install a universal forwarder on the machine that you want to collect the AD data.
- Install the Splunk Cloud universal forward credentials package onto the machine.
Configure file monitoring with inputs.conf
- On the machine that runs Splunk software, open a shell or command prompt.
- Change the listed directory to the $SPLUNK_HOME/etc/system/local directory.
- If the inputs.conf file doesn't exist, create the file.
- Open inputs.conf for editing with a text editor.
- Add a stanza that references the files or directories that you want to monitor.
For example, to monitor the /var/log/messages file on a *nix system, use this specification:
[monitor:///var/log/messages] disabled = 0
To monitor the C:\Windows\System32\WindowsUpdate.log file on a Windows system, use this specification:
[monitor://C:\Windows\System32\WindowsUpdate.log] disabled = 0
- (Optional) Add settings that further configure the input, depending on what you want the input to do. See Configuration settings later in this topic, or see inputs.conf in the Splunk Enterprise Admin Manual for additional settings.
[monitor://path/to/file] disabled = 0 setting1 = value setting2 = value ...
- Save the inputs.conf file and close it.
- Either restart the Splunk platform or reload the configuration by running the following command. The Splunk platform prompts you for credentials if you reload the configuration.
./splunk _internal call /services/data/inputs/monitor/_reload -auth
You can use the following settings in both
batch input stanzas.
||Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. The Splunk platform prepends the
||The IP address or fully qualified domain name of the host where the data originated.|
||Sets the index where events from this input are stored. The Splunk platform prepends the
For more information about the index field, see How indexing works in the Splunk Enterprise Managing Indexers and Clusters manual.
||Sets the sourcetype key or field for events from this input. This setting explicitly declares the source type for this data, as opposed to letting the Splunk platform determine it automatically. Declaring the sourcetype is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.
Sets the sourcetype key initial value. The Splunk platform uses the key during parsing and indexing to set the source type field and uses the source type field during searching. The Splunk platform prepends the
For more information about source types, see Why source types matter.
|The Splunk platform picks a source type based on various aspects of the data. There is no default.|
||Specifies where the input processor deposits the events that it reads. Set to
||Specifies a comma-separated list of tcpout group names. Use this setting to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data.
Define the tcpout group names in the outputs.conf file in
|The groups present in |
||A regular expression that extracts the host from the file name of each input. Specifically, the Splunk platform uses the first group of the regular expression as the host.||The default "|
||Sets the segment of the path as the host, using
||The default "|
Monitor input stanzas configure the Splunk platform to watch all files in the
<path> or the
<path> itself if it represents a single file. You must specify the input type before the path, so add three forward slashes in the path if the path includes the root directory on *nix machines.
You can use wildcards for the path. See Specify input paths with wildcards.
[monitor://<path>] <setting1> = <val1> <setting2> = <val2> ...
The following are additional settings you can use when defining monitor input stanzas:
||Sets the source field for events from this input. You can use this setting when using the
The Splunk platform prepends the
|The input file path, except in the case of |
||Forces the Splunk platform to index files that have matching cyclic redundancy checks (CRCs). By default, the software performs CRCs only against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs.)
If set, the Splunk platform adds
Use caution with this setting for rolling log files. This setting can lead to the log file being re-indexed after it has rolled.
This setting is case-sensitive.
||Causes the input to stop checking files for updates if the file modification time has passed the
The Splunk platform doesn't index files whose modification time falls outside
You must specify
||If set to
||If set, the Splunk platform monitors files whose names match the specified regular expression.||N/A|
||If set, the Splunk platform doesn't monitor files whose names match the specified regular expression.||N/A|
||If set to
Use this setting for monitoring files on Windows, and for Internet Information Server (IIS) logs.
Use caution with this setting, as it increases load and slows down indexing.
||If set to
||The modification time delta required before the Splunk platform can close a file on end-of-file. This setting tells the system not to close files that have been updated in the past
||If set to
MonitorNoHandle input monitors files without using Windows file handles. This input allows Splunk software to read special Windows log files such as the DNS debug server log. There are several limitations when using this input:
MonitorNoHandleinput stanza works on Windows systems only.
MonitorNoHandleinput stanza monitors only a single file.
- You can't use wildcards in the file or directory path.
- You can't monitor directories using a
MonitorNoHandleinput stanza reads only new data written to the monitored file. It doesn't ingest data already written to the file.
- A file monitored using
MonitorNoHandlehas the source metadata set to
MonitorNoHandleby default. To specify another source, you must define it using the
sourcesetting in the inputs.conf file stanza.
For an example of a
MonitorNoHandle stanza, see MonitorNoHandle, single Windows file.
Use batch to set up a one-time, destructive input of data from a source.
For continuous, nondestructive inputs, use the
monitor input. The Splunk platform deletes data that it has indexed with the
[batch://<path>] move_policy = sinkhole <setting1> = <val1> <setting2> = <val2> ...
When you define batch inputs, you must include the
move_policy = sinkhole setting. This setting loads the file destructively. Don't use the batch input type for files that you don't want to delete after indexing.
To ensure that the Splunk platform indexes new events when you copy over an existing file with new contents, set the
CHECK_METHOD = modtime setting in the props.conf file for the input source. This setting checks the modification time of the file and re-indexes it when the time changes. The Splunk platform indexes the entire file, which can result in duplicate events. For information about the props.conf file, see props.conf.
Examples of monitor input stanzas
Single *nix file
This example stanza configures the Splunk platform to index the single /var/log/messages file:
[monitor:///var/log/messages] disabled = 0 sourcetype = unixlog
Single Windows directory
This Windows example configures the Splunk platform to monitor the C:\Windows\Logs directory and all the files in it:
[monitor://C:\Windows\Logs] disabled = 0
Single Windows directory with spaces in filename
This Windows example configures the Splunk platform to monitor the C:\Program Files\VMWare directory and all the files in it:
[monitor://C:\Program Files\VMWare] disabled = 0
Multiple Windows directories
This Windows example tells the Splunk platform to monitor all of the directories in C:\Windows\Debug:
[monitor://C:\Windows\Debug\*] disabled = 0
Multiple *nix directories with a wildcard
This example configures the Splunk platform to monitor directories like /apache/foo/log and /apache/bar/log:
Multiple *nix files in one directory with a wildcard
This *nix example configures the Splunk platform to monitor multiple files in one directory, such as /apache/*.log:
MonitorNoHandle, single Windows file
This single Windows file example is from the Splunk Add-on for Microsoft Windows on Splunkbase:
###### Monitor Inputs for DNS ###### [MonitorNoHandle://$WINDIR\System32\Dns\dns.log] sourcetype=MSAD:NT6:DNS disabled=0
This batch example loads and deletes all files from the system/flight815/ directory:
[batch://system/flight815/*] move_policy = sinkhole
Monitor Splunk Enterprise files and directories with the CLI
Specify input paths with wildcards
This documentation applies to the following versions of Splunk® Enterprise: 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.2.0