
Include or exclude specific incoming data
You can use allow list and deny list rules to specify which files to consume or exclude when you monitor a directory or set of directories. You can also apply these settings to batch
type monitoring inputs. When you define an allow list, Splunk Enterprise only indexes the files you specify. When you define a deny list, the platform ignores the specified files and processes all other files. You define these filters in the input stanza in the inputs.conf
configuration file. If you want to apply the same filters across all of the forwarders in a Splunk platform deployment, you can set up a deployment server to perform this task.
It is not necessary to define both an allow list and a deny list in a configuration stanza. The settings are independent. If you do define both filters and a file matches them both, Splunk Enterprise does not index that file, as the blacklist
filter overrides the whitelist
filter.
The list rules use the regular expression syntax to define the match on the file name or path. They must be contained within a configuration stanza, for example [monitor://<path>]
. The Splunk platform ignores filter lists that are not inside a stanza. When you define filter entries, you must use exact regular expression syntax.
To learn more about regular expressions and how to build them, see the Regular-expressions.info (http://regular-expressions.info) website.
Route and filter data
Instead of including or excluding your data inputs, you can filter specific events and send them to different queues or indexes.
Include files
- Add the following line to your
monitor
stanza in the/local/inputs.conf
file for the app context that you defined the input.
whitelist = <your_custom regex>
For example, to monitor only files with the
.log
extension:[monitor:///mnt/logs] whitelist = \.log$
Include multiple files
You can include multiple files in one line, using the "|" (pipe, or "OR") operator. For example, to include filenames that contain query.log OR my.log
.
whitelist = query\.log$|my\.log$
Or, you can include only files that match exactly.
whitelist = /query\.log$|/my\.log$
The "$" anchors the regular expression to the end of the line. There is no space before or after the "|" operator.
Exclude files
- Add the following line to your
monitor
stanza in the/local/inputs.conf
configuration file for the app context in which you defined the input.
blacklist = <your_custom_regex>
If you create a blacklist
entry for each file you want to ignore, Splunk Enterprise activates only the last filter.
Example: Exclude only files with a .txt extension
To ignore and not monitor only files with the .txt
extension:
[monitor:///mnt/logs] blacklist = \.txt$
Example 2: Exclude files with either a .txt or .gz extension
To ignore and not monitor all files with either the .txt
extension OR
the .gz
extension:
[monitor:///mnt/logs] blacklist = \.(?:txt|gz)$
Example 3: Exclude an entire directory
To ignore entire directories beneath a monitor input:
[monitor:///mnt/logs] blacklist = archive|historical|\.bak$
This example configures Splunk Enterprise to ignore all files under /mnt/logs/ within the "archive" or "historical" directories, and all files ending in *.bak.
Example 4: Blacklist a file whose name contains a string
To ignore files whose names contain a specific string:
[monitor:///mnt/logs] blacklist = 2009022[89]file\.txt$
This example ignores the webserver20090228file.txt
and webserver20090229file.txt
files under /mnt/logs/
.
Example 5: Exclude Windows Event Code #4662 events whose "Message" field contains a specific value
To ignore Windows Event Code #4662 events whose Message field contains events with the value Account Name: "example account"
:
[WinEventLog:Security] blacklist1 = EventCode = "4662" Message = "Account Name:\s+(example account)"
This example ignores Windows Security events with Event Code 4662
and Message values with Account Name: example account
.
PREVIOUS Specify input paths with wildcards |
NEXT How Splunk Enterprise handles log file rotation |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.1.0, 8.1.1
Feedback submitted, thanks!