Splunk Cloud

Getting Data In

Acrobat logo Download manual as PDF

This documentation does not apply to the most recent version of SplunkCloud. Click here for the latest version.
Acrobat logo Download topic as PDF

Monitor changes to your file system

This feature has been deprecated.
This feature has been deprecated as of Splunk Enterprise version 5.0. This means that although it continues to function in this version of Splunk software, it might be removed in a future version. As an alternative, you can use the auditd daemon on *nix systems and monitor output from the daemon.

For a list of all deprecated features, see the topic Deprecated features in the Release Notes.

The file system change monitor tracks changes in your file system. The monitor watches a directory you specify and generates an event when that directory undergoes a change. It is completely configurable and can detect when any file on the system is edited, deleted, or added (not just files that are specific to the Splunk platform).

For example, you can configure the file system change monitor to watch the /etc/sysconfig/ directory on a *nix machine and alert you any time the system configurations change.

How the file system change monitor works

The file system change monitor detects changes using:

  • file modification date/time
  • group ID
  • user ID
  • file mode (read/write attributes, etc.)
  • optional SHA256 hash of file contents

You can configure the following features of the file system change monitor:

  • include files using regular expressions
  • exclude using regular expressions
  • directory recursion
    • including symbolic link traversal
    • scanning multiple directories, each with their own polling frequency
  • cryptographic signing
    • creates a distributed audit trail of file system changes
  • indexing entire file as an event on add/change
    • size cutoffs for sending entire file and/or hashing
  • all change events indexed by, and searchable through, Splunk Enterprise

By default, the file system change monitor generates audit events whenever the contents of $SPLUNK_HOME/etc/ are changed, deleted, or added to. When you start the Splunk platform for the first time, it generates an audit event for each file in the $SPLUNK_HOME/etc/ directory and all subdirectories. Afterward, any change in configuration (regardless of origin) generates an audit event for the affected file. If you have configured signedaudit=true, the platform indexes the file system change into the audit index (index=_audit). If signedaudit is not turned on, by default, Splunk Enterprise writes the events to the main index unless you specify another index.

The file system change monitor does not track the user name of the account that executes the change, only that a change has occurred. For user-level monitoring, consider using native operating system audit tools, which have access to this information.

Do not configure the file system change monitor to monitor your root file system. This can be dangerous and time-consuming if you also enable directory recursion.

Configure the file system change monitor

Configure the file system change monitor using the inputs.conf configuration file. There is no support for configuring the file system change monitor in Splunk Web. You must restart the Splunk platform instance any time you make changes to a [fschange] stanza. For more information, see inputs.conf in the Admin Manual.

  1. Open inputs.conf.
  2. Add [fschange:<directory>] stanzas to specify files or directories that Splunk software should monitor for changes.
  3. Save the inputs.conf file and close it.
  4. Restart the Splunk platform instance. File system change monitoring begins immediately.

If you want to use this feature with forwarding, follow these guidelines:

To use the file system change monitor to watch any directory, add or edit an [fschange] stanza to an inputs.conf configuration file in $SPLUNK_HOME/etc/system/local/ or your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see About configuration files in the Admin Manual.

Syntax

Here is the syntax for the [fschange] stanza:

[fschange:<directory or file to monitor>]
<setting1> = <val1>
<setting2> = <val2>
...

* The Splunk platform monitors all adds, updates, and deletions to the directory and its subdirectories. Any change generates an event that the platform indexes. By default, is $SPLUNK_HOME/etc/.

Settings

All settings are optional. Here is the list of available attributes:

Setting Description Default
index=<indexname> The index where all generated events should be stored. main (unless you have turned on audit event signing).
recurse=<true | false> Whether or not to recurse all directories within the directory specified in [fschange]. Set to true to recurse all subdirectories and false to specify only the current directory. true
followLinks=<true | false> Whether or not the file system change monitor follows symbolic links. Set to true to follow symbolic links and false not to follow symbolic links. false

Caution: If you are not careful when setting followLinks, file system loops can occur.

pollPeriod=N Check this directory for changes every N seconds. 3600 seconds

If you make a change, the file system audit events could take anywhere between 1 and 3600 seconds to be generated and become available in audit search.

hashMaxSize=N Calculate a SHA1 hash for every file that is less than or equal to N size in bytes.

This hash can be used as an additional method for detecting change in the file/directory.

-1 (no hashing used for change detection).
signedaudit=<true | false> Send cryptographically signed add/update/delete events.

Set to true to generate events in the _audit index. Set to false if you're setting the index attribute.

When you set signedaudit to true, confirm that auditing is enabled in the audit.conf configuration file. For more information, see the audit.conf specification file in the ''Admin Manual''.

false
fullEvent=<true | false> * Send the full event if an add or update change is detected.
  • Further qualified by the sendEventMaxSize attribute.
false
sendEventMaxSize=N * Only send the full event if the size of the event is less than or equal to N bytes.

This limits the size of indexed file data.

-1 (unlimited.)
sourcetype = <string> Set the source type for events from this input.

"sourcetype::" is prepended to <string>.

audittrail (if signedaudit=true) or fs_notification (if signedaudit=false)
filesPerDelay = <integer> Injects a delay specified by delayInMills after processing <integer> files. This throttles file system monitoring so it does not consume as much CPU. n/a
delayInMills = <integer> The delay in milliseconds to use after processing every <integer> files as specified in filesPerDelay. This is used to throttle file system monitoring so it does not consume as much CPU.
filters=<filter1>,<filter2>,...<filterN> Each of these filters will apply from left to right for each file or directory that is found during the monitors poll cycle. See the next section for information on defining filters. n/a

Define a filter

To define a filter to use with the filters setting, add a [filter...] stanza as follows:

[filter:blacklist:backups] 
regex1 = .*bak
regex2 = .*bk

[filter:whitelist:code] 
regex1 = .*\.c 
regex2 = .*\.h 
 
[fschange:/etc] 
filters = backups,code 

The filter type determines whether or not the filter includes or excludes files based on the regular expressions that you specify.

  • [filter:whitelist:<filtername>] represents an allow list.
  • [filter:blacklist:<filtername>] represents a deny list.

The following list describes how the Splunk platform handles fschange allow- and deny list logic:

  • The events run down through the list of filters until they reach their first match.
  • If the first filter to match an event is an allow list, then Splunk software indexes the event.
  • If the first filter to match an event is a deny list, the filter prevents the event from getting indexed.
  • If an event reaches the end of the chain with no matches, then Splunk software indexes the event. This means that there is an implicit "all pass" filter built in.

To default to a situation where the Splunk platform does not index events if they don't match an allow list explicitly, end the chain with a deny list that matches all remaining events.

For example:

...
filters = <filter1>, <filter2>, ... terminal-denylist

[filter:blacklist:terminal-denylist] 
regex1 = .?

If you exclude a directory including a terminal deny list at the end of a series of allow lists, then the Splunk platform excludes all its subfolders and files, as they do not pass any allow list. To accommodate this, include all folders and subfolders that you want explicitly ahead of the deny list items in your filters.

Example of explicit inclusion and terminal exclusion

This configuration monitors files in the specified directory with the extensions .config, .xml, .properties, and .log and ignores all others.

In this example, a directory could be excluded. If this is the case, Splunk Enterprise excludes all of its subfolders and files as well. Only files in the specified directory would be monitored.

[filter:whitelist:configs]
regex1 = .*\.config 
regex2 = .*\.xml 
regex3 = .*\.properties 
regex4 = .*\.log
 
[filter:blacklist:terminal-denylist]
regex1 = .?
 
[fschange:/var/apache] 
index = sample 
recurse = true 
followLinks = false 
signedaudit = false 
fullEvent = true 
sendEventMaxSize = 1048576 
delayInMills = 1000 
filters = configs,terminal-denylist 

Use with a universal forwarder

To forward file system change monitor events from a universal forwarder, you must configure the following settings and values:

  • signedaudit = false
  • index=_audit
[fschange:<directory or file to monitor>]
signedaudit = false
index=_audit

With this workaround, the universal forwarder indexes file system change monitor events into the _audit index with sourcetype set to fs_notification and source set to fschangemonitor, instead of the default value of audittrail for both sourcetype and source .

Last modified on 12 June, 2020
PREVIOUS
Monitor First In, First Out (FIFO) queues
  NEXT
Get data from APIs and other remote data interfaces through scripted inputs

This documentation applies to the following versions of Splunk Cloud: 7.2.4, 7.2.6, 7.2.7, 7.2.8, 7.2.9


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters