Monitor files and directories with inputs.conf

You can use the inputs.conf file to monitor files and directories with the Splunk platform. The inputs.conf file provides the most configuration options for setting up a file monitor input. If you use Splunk Cloud Platform, you can use either Splunk Web or a forwarder to configure file monitoring inputs.

To configure an input, add a stanza to the inputs.conf file in the $SPLUNK_HOME/etc/system/local/ directory or your own custom application directory in $SPLUNK_HOME/etc/apps/. These locations are on the machine that runs Splunk Enterprise or the forwarder. To learn more about the inputs.conf file, see inputs.conf in the Splunk Enterprise Admin Manual.

You can configure multiple settings in an input stanza. If you don't specify a value for a setting, the Splunk platform uses the default for that setting. You can find the defaults for settings in the $SPLUNK_HOME/etc/system/default/inputs.conf directory.

For more information about configuration files, see About configuration files in the Splunk Enterprise Admin Manual.

Configure a forwarder to send data to Splunk Cloud Platform

If you want to send Active Directory (AD) data to Splunk Cloud Platform, you must install and configure a forwarder before you begin making edits to configuration files on the forwarder.

1. Install a universal forwarder on the machine that you want to collect the AD data.
2. Install the Splunk Cloud Platform universal forward credentials package onto the machine.

Configure file monitoring with inputs.conf

1. On the machine that runs Splunk software, open a shell or command prompt.
2. Change the listed directory to the $SPLUNK_HOME/etc/system/local directory.
3. If the inputs.conf file doesn't exist, create the file.
4. Open inputs.conf for editing with a text editor.
5. Add a stanza that references the files or directories that you want to monitor. For example, to monitor the /var/log/messages file on a *nix system, use this specification:

   [monitor:///var/log/messages]
   disabled = 0
   

   To monitor the C:\Windows\System32\WindowsUpdate.log file on a Windows system, use this specification:

   [monitor://C:\Windows\System32\WindowsUpdate.log]
   disabled = 0
   

6. (Optional) Add settings that further configure the input, depending on what you want the input to do. See Configuration settings later in this topic, or see inputs.conf in the Splunk Enterprise Admin Manual for additional settings.

   [monitor://path/to/file]
   disabled = 0
   setting1 = value
   setting2 = value
   ...
   

7. Save the inputs.conf file and close it.
8. Either restart the Splunk platform or reload the configuration by running the following command. The Splunk platform prompts you for credentials if you reload the configuration.

   ./splunk _internal call /services/data/inputs/monitor/_reload -auth
   

Configuration settings

You can use the following settings in both monitor and batch input stanzas.

Setting	Description	Default
host = <string>	Sets the host key to a static initial value for this stanza. The input processor uses the key during parsing and indexing to set the host field and uses the field during searching. The Splunk platform prepends the <string> with host::.	The IP address or fully qualified domain name of the host where the data originated.
index = <string>	Sets the index where events from this input are stored. The Splunk platform prepends the <string> with index::. For more information about the index field, see How indexing works in the Splunk Enterprise Managing Indexers and Clusters manual.	The main index or whatever you set the default index to.
sourcetype = <string>	Sets the sourcetype key or field for events from this input. This setting explicitly declares the source type for this data, as opposed to letting the Splunk platform determine it automatically. Declaring the sourcetype is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing. Sets the sourcetype key initial value. The Splunk platform uses the key during parsing and indexing to set the source type field and uses the source type field during searching. The Splunk platform prepends the <string> with sourcetype::. For more information about source types, see Why source types matter.	The Splunk platform picks a source type based on various aspects of the data. There is no default.
queue = parsingQueue | indexQueue	Specifies where the input processor deposits the events that it reads. Set to parsingQueue to apply the props.conf file and other parsing rules to your data. Set to indexQueue to send your data directly into the index.	parsingQueue
_TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>,...	Specifies a comma-separated list of tcpout group names. Use this setting to selectively forward your data to specific indexers by specifying the tcpout groups that the forwarder should use when forwarding the data. Define the tcpout group names in the outputs.conf file in [tcpout:<tcpout_group_name>] stanzas.	The groups present in defaultGroup in [tcpout] stanza in the outputs.conf file.
host_regex = <regular expression>	A regular expression that extracts the host from the file name of each input. Specifically, the Splunk platform uses the first group of the regular expression as the host.	The default "host =" setting, if the regular expression fails to match.
host_segment = <integer>	Sets the segment of the path as the host, using <integer> to determine the segment. For example, if host_segment = 2, host becomes the second segment of the path. Path segments are separated by the forward slash ( / ) character.	The default "host =" setting, if the value is not an integer or is less than 1.

Monitor syntax

Monitor input stanzas configure the Splunk platform to watch all files in the <path> or the <path> itself if it represents a single file. You must specify the input type before the path, so add three forward slashes in the path if the path includes the root directory on *nix machines.

You can use wildcards for the path. See Specify input paths with wildcards.

[monitor://<path>]
<setting1> = <val1>
<setting2> = <val2>
...

The following are additional settings you can use when defining monitor input stanzas:

Setting	Description	Default
source = <string>	Sets the source field for events from this input. You can use this setting when using the MonitorNoHandle input and want to set the source to the name of the file you're monitoring. Otherwise, don't override unless absolutely necessary. Consider using source types, tagging, and search wildcards instead. The input layer provides a more accurate string to aid in problem analysis and investigation by accurately recording the file from which the data was retrieved. The Splunk platform prepends the <string> with source::.	The input file path, except in the case of MonitorNoHandle, where the default is MonitorNoHandle.
crcSalt = <string>	Forces the Splunk platform to index files that have matching cyclic redundancy checks (CRCs). By default, the software performs CRCs only against the first few lines of a file. This behavior prevents indexing of the same file twice, even though you might have renamed it, such as with rolling log files. However, because the CRC counts only the first few lines of the file, it is possible for legitimately different files to have matching CRCs.) If set, the Splunk platform adds string to the CRC. If set to <SOURCE>, the Splunk platform adds the full source path to the CRC. Adding <SOURCE> ensures that each file being monitored has a unique CRC. Use caution with this setting for rolling log files. This setting can lead to the log file being re-indexed after it has rolled. This setting is case-sensitive.	N/A
ignoreOlderThan = <time_window>	Causes the input to stop checking files for updates if the file modification time has passed the <time_window> threshold. Stopping the file checking improves the speed of file tracking operations when you are monitoring directory hierarchies with large numbers of historical files. For example, when active log files share a directory with old files that no longer get writes. The Splunk platform doesn't index files whose modification time falls outside <time_window> when it first attempts to monitor the file. You must specify <number><unit>. For example, 7d indicates one week. Valid units are d for days, h for hours, m for minutes, and s for seconds.	0 (disabled)
followTail = 0|1	If set to 1, monitoring begins at the end of the file, much like *nix tail -f. This setting applies only to files the first time the Splunk platform attempts to monitor them. After that, the Splunk platform keeps track of the file using its internal file position records.	0
whitelist = <regular expression>	If set, the Splunk platform monitors files whose names match the specified regular expression.	N/A
blacklist = <regular expression>	If set, the Splunk platform doesn't monitor files whose names match the specified regular expression.	N/A
alwaysOpenFile = 0 | 1	If set to 1, the Splunk platform opens a file to check if it's been indexed. This setting is useful only for files that don't update their modification time. Use this setting for monitoring files on Windows, and for Internet Information Server (IIS) logs. Use caution with this setting, as it increases load and slows down indexing.	N/A
recursive = true|false	If set to false, the Splunk platform doesn't look into subdirectories that it finds within a monitored directory.	true
time_before_close = <integer>	The modification time delta required before the Splunk platform can close a file on end-of-file. This setting tells the system not to close files that have been updated in the past <integer> seconds.	3
followSymlink = true|false	If set to false, the Splunk platform ignores symbolic links that it finds within a monitored directory.	true

MonitorNoHandle syntax

The MonitorNoHandle input monitors files without using Windows file handles. This input allows Splunk software to read special Windows log files such as the DNS debug server log. There are several limitations when using this input:

• The MonitorNoHandle input stanza works on Windows systems only.
• The MonitorNoHandle input stanza monitors only a single file.
• You can't use wildcards in the file or directory path.
• You can't monitor directories using a MonitorNoHandle stanza.
• The MonitorNoHandle input stanza reads only new data written to the monitored file. It doesn't ingest data already written to the file.
• A file monitored using MonitorNoHandle has the source metadata set to MonitorNoHandle by default. To specify another source, you must define it using the source setting in the inputs.conf file stanza.

For an example of a MonitorNoHandle stanza, see MonitorNoHandle, single Windows file.

Batch syntax

Use batch to set up a one-time, destructive input of data from a source.

For continuous, nondestructive inputs, use the monitor input. The Splunk platform deletes data that it has indexed with the batch input.

[batch://<path>]
move_policy = sinkhole
<setting1> = <val1>
<setting2> = <val2>
...

When you define batch inputs, you must include the move_policy = sinkhole setting. This setting loads the file destructively. Don't use the batch input type for files that you don't want to delete after indexing.

To ensure that the Splunk platform indexes new events when you copy over an existing file with new contents, set the CHECK_METHOD = modtime setting in the props.conf file for the input source. This setting checks the modification time of the file and re-indexes it when the time changes. The Splunk platform indexes the entire file, which can result in duplicate events. For information about the props.conf file, see props.conf.

Examples of monitor input stanzas

Single *nix file

This example stanza configures the Splunk platform to index the single /var/log/messages file:

[monitor:///var/log/messages]
disabled = 0
sourcetype = unixlog

Single Windows directory

This Windows example configures the Splunk platform to monitor the C:\Windows\Logs directory and all the files in it:

[monitor://C:\Windows\Logs]
disabled = 0

Single Windows directory with spaces in filename

This Windows example configures the Splunk platform to monitor the C:\Program Files\VMWare directory and all the files in it:

[monitor://C:\Program Files\VMWare]
disabled = 0

Multiple Windows directories

This Windows example tells the Splunk platform to monitor all of the directories in C:\Windows\Debug:

[monitor://C:\Windows\Debug\*]
disabled = 0

Multiple *nix directories with a wildcard

This example configures the Splunk platform to monitor directories like /apache/foo/log, /apache/bar/log, and /apache/foo/bar/log:

[monitor:///apache/.../log]

Multiple *nix files in one directory with a wildcard

This *nix example configures the Splunk platform to monitor multiple files in one directory, such as /apache/*.log:

[monitor:///apache/*.log]

MonitorNoHandle, single Windows file

This single Windows file example is from the Splunk Add-on for Microsoft Windows on Splunkbase:

###### Monitor Inputs for DNS ######
[MonitorNoHandle://$WINDIR\System32\Dns\dns.log]
sourcetype=MSAD:NT6:DNS
disabled=0

Batch

This batch example loads and deletes all files from the system/flight815/ directory:

[batch://system/flight815/*]
move_policy = sinkhole