Monitor files and directories with inputs.conf

You can use inputs.conf to monitor files and directories with Splunk Enterprise. Inputs.conf provides the most configuration options for setting up a file monitor input. For Splunk Cloud, use Splunk Web to configure file monitoring inputs instead.

To configure an input to Splunk Enterprise, add a stanza to inputs.conf in $SPLUNK_HOME/etc/system/local/ or your own custom application directory in $SPLUNK_HOME/etc/apps/. These locations are on the machine that runs Splunk Enterprise.

You can configure multiple settings in an input stanza. If you do not specify a value for a setting, Splunk Enterprise uses the default for that setting. You can find the defaults for settings in $SPLUNK_HOME/etc/system/default/inputs.conf.

For more information about configuration files, see About configuration files.

Configure monitoring of files with inputs.conf

1. On the machine that runs Splunk Enterprise, open a shell or command prompt.
2. Change to the $SPLUNK_HOME/etc/system/local directory.
3. (Optional) If inputs.conf does not exist, create the file.
4. Open inputs.conf for editing with a text editor.
5. Add a stanza that references the files or directories that you want to monitor. For example, to monitor the /var/log/messages file on a *nix system, specify:

   [monitor:///var/log/messages]
   disabled = 0
   

   To monitor C:\Windows\System32\WindowsUpdate.log on a Windows system, specify:

   [monitor://C:\Windows\System32\WindowsUpdate.log]
   disabled = 0
   

6. (Optional) Add settings that further configure the input, depending on what you want the input to do. See Configuration settings later in this topic, or review the inputs.conf configuration specification file in the Admin Manual for additional settings.

   [monitor://path/to/file]
   disabled = 0
   setting1 = value
   setting2 = value
   ...
   

7. Save the inputs.conf file and close it.
8. Either restart Splunk Enterprise or reload the configuration by running the following command. Splunk Enterprise prompts you for credentials if you reload the configuration.

   ./splunk _internal call /services/data/inputs/monitor/_reload -auth
   

Configuration settings

You can use the following settings in both monitor and batch input stanzas.

Setting	Description	Default
host = <string>	Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. Splunk Enterprise prepends the <string> with host::.	the IP address or fully-qualified domain name of the host where the data originated.
index = <string>	Sets the index where events from this input will be stored. Splunk Enterprise prepends the <string> with index::. For more information about the index field, see How indexing works in the Managing Indexers and Clusters manual.	main or whatever you set the default index to
sourcetype = <string>	Sets the sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to letting Splunk Enterprise determine it automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing. Sets the sourcetype key initial value. Splunk Enterprise uses the key during parsing and indexing to set the source type field and uses the source type field during searching. Splunk Enterprise prepends the <string> with sourcetype::. For more information about source types, see Why source types matter.	Splunk Enterprise picks a source type based on various aspects of the data. There is no default.
queue = parsingQueue | indexQueue	Specifies where the input processor should deposit the events that it reads. Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.	parsingQueue
_TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>,...	Specifies a comma-separated list of tcpout group names. Use this setting to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data. Define the tcpout group names in outputs.conf in [tcpout:<tcpout_group_name>] stanzas.	the groups present in 'defaultGroup' in [tcpout] stanza in outputs.conf
host_regex = <regular expression>	A regular expression that extracts host from the file name of each input. Specifically, Splunk Enterprise uses the first group of the regular expression as the host.	the default "host =" setting, if the regular expression fails to match
host_segment = <integer>	Sets the segment of the path as the host, using <integer> to determine the segment. For example, if host_segment = 2, host becomes the second segment of the path. Path segments are separated by the '/' character.	the default "host =" setting, if the value is not an integer, or is less than 1

Monitor syntax

Monitor input stanzas configure Splunk Enterprise to watch all files in the <path> or the <path> itself if it represents a single file. You must specify the input type before the path, so add three slashes in the path if the path includes the root directory on *nix machines.

You can use wildcards for the path. See Specify input paths with wildcards.

[monitor://<path>]
<setting1> = <val1>
<setting2> = <val2>
...

The following are additional settings you can use when defining monitor input stanzas:

Setting	Description	Default
source = <string>	Sets the source field for events from this input. You can use this setting when using the MonitorNoHandle input and want to set the source to the name of the file you are monitoring. Otherwise, do not override unless absolutely necessary. Consider use of source types, tagging, and search wildcards instead. The input layer usually provides a more accurate string to aid in problem analysis and investigation by accurately recording the file from which the data was retrieved. Splunk Enterprise prepends the <string> with source::.	the input file path (except in the case of MonitorNoHandle, where it is MonitorNoHandle)
crcSalt = <string>	Forces Splunk Enterprise to index files that have matching cyclic redundancy checks (CRCs). By default, the software only performs CRCs against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs.) If set, Splunk Enterprise adds string to the CRC. If set to <SOURCE>, Splunk Enterprise adds the full source path to the CRC. This ensures that each file being monitored has a unique CRC. Use caution with this setting for rolling log files. It can lead to the log file being re-indexed after it has rolled. This setting is case sensitive.	N/A
ignoreOlderThan = <time_window>	Causes the input to stop checking files for updates if the file modification time (modtime) has passed the <time_window> threshold. This improves the speed of file tracking operations when monitoring directory hierarchies with large numbers of historical files (for example, when active log files share a directory with old files that no longer get writes). Splunk Enterprise does not index files whose modification time falls outside <time_window> when it first attempts to monitor the file. You must specify <number><unit>. For example, "7d" indicates one week. Valid units are "d" (days), "h" (hours), "m" (minutes), and "s" (seconds).	0 (disabled)
followTail = 0|1	If set to 1, monitoring begins at the end of the file (like *nix tail -f). This only applies to files the first time Splunk Enterprise attempts to monitor them. After that, Splunk Enterprise keeps track of the file using its internal file position records.	0
whitelist = <regular expression>	If set, Splunk Enterprise only monitors files whose names match the specified regular expression.	N/A
blacklist = <regular expression>	If set, Splunk Enterprise does NOT monitor files whose names match the specified regular expression.	N/A
alwaysOpenFile = 0 | 1	If set to 1, Splunk Enterprise opens a file to check if it has already been indexed. This is only useful for files that don't update their modification time. Use this setting for monitoring files on Windows, and for Internet Information Server (IIS) logs. Caution: Use of this setting increases load and slows down indexing.	N/A
recursive = true|false	If set to false, Splunk Enterprise does not look into subdirectories that it finds within a monitored directory.	true
time_before_close = <integer>	The modification time delta required before Splunk Enterprise can close a file on End-of-file (EOF). Tells the system not to close files that have been updated in the past <integer> seconds.	3
followSymlink = true|false	If false, Splunk Enterprise ignores symbolic links that it finds within a monitored directory.	true

MonitorNoHandle syntax

On Windows systems only, the MonitorNoHandle input monitors files without using Windows file handles. This lets you read special log files like the Windows DNS server log files.

You must specify a valid path to a file when you use the MonitorNoHandle input stanza. You cannot specify a directory. If you specify a file that already exists, Splunk Enterprise does not index the existing data in the file. It only indexes new data that the system writes to the file.

MonitorNoHandle sets the source for files you monitor to MonitorNoHandle.To specify the file name as the source, you must use the source setting in the stanza for the MonitorNoHandle input for the file.

You can only configure MonitorNoHandle on a Windows machine with inputs.conf or the Command Line Interface (CLI). You cannot configure it in Splunk Web or on a *nix machine.

[MonitorNoHandle://<path>]
disabled=0
source = <path>
<setting1> = <val1>
<setting2> = <val2>
...

Batch syntax

Use batch to set up a one-time, destructive input of data from a source.

For continuous, non-destructive inputs, use monitor. Splunk enterprise deletes data that it has indexed with the batch monitor.

[batch://<path>]
move_policy = sinkhole
<setting1> = <val1>
<setting2> = <val2>
...

When you define batch inputs, you must include the setting move_policy = sinkhole. This loads the file destructively. Do not use the batch input type for files that you do not want to delete after indexing.

To ensure that Splunk Enterprise indexes new events when you copy over an existing file with new contents, set the CHECK_METHOD = modtime setting in props.conf for the input source. This checks the modification time of the file and re-indexes it when the time changes. Splunk Enterprise indexes the entire file, which can result in duplicate events.

Examples of monitor input stanzas

Single *nix file

This example stanza configures Splunk Enterprise to index the single file, /var/log/messages.

[monitor:///var/log/messages]
disabled = 0
sourcetype = unixlog

Single Windows directory

This Windows example configures Splunk Enterprise to monitor the directory, C:\Windows\Logs. and all the files in it.

[monitor://C:\Windows\Logs]
disabled = 0

Single Windows directory with spaces in filename

This Windows example configures Splunk Enterprise to monitor the directory, C:\Program Files\VMWare. and all the files in it.

[monitor://C:\Program Files\VMWare]
disabled = 0

Multiple Windows directories

This Windows example tells Splunk Enterprise to monitor all of the directories in C:\Windows\Debug.

[monitor://C:\Windows\Debug\*]
disabled = 0

Multiple *nix directories with a wildcard

This example configures Splunk Enterprise to monitor directories like /apache/foo/log, /apache/bar/log, etc.

[monitor:///apache/.../log]

Multiple *nix files in one directory with a wildcard

This *nix example configures Splunk Enterprise to monitor multiple files in one directory, such as /apache/*.log.

[monitor:///apache/*.log]

MonitorNoHandle, single Windows file

This Windows example uses the MonitorNoHandle input to monitor a file that Windows has open for writing, such as C:\Windows\System32\DNS\DNS.log.

[MonitorNoHandle://<path>]
disabled = 0
source = <path>
<setting1> = <val1>
<setting2> = <val2>
...

Batch

This batch example loads and deletes all files from the directory system/flight815/.

[batch://system/flight815/*]
move_policy = sinkhole