Splunk® Enterprise

Getting Data In

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Specify input paths with wildcards

This topic discusses how to specify wildcards in a path in inputs.conf. It is only relevant when using inputs.conf to specify inputs, as described in "Edit inputs.conf" in this manual.

Important: Input path specifications in inputs.conf don't use regular expressions (regexes) but rather Splunk-defined wildcards.

Wildcard overview

A wildcard is a character that you can substitute for one or more unspecified characters when searching text or selecting multiple files or directories. In Splunk Enterprise, you can use wildcards to specify your input path for monitored input.

Wildcard Description Regex equivalent Example(s)
... The ellipsis wildcard recurses through directories and any number of levels of subdirectories to find matches.

Note: if the wildcard is followed by any folder separator (//var/log/.../file) it will not match the first folder level, only sub folders.

.* /foo/.../bar.log matches the files /foo/1/bar.log, /foo/2/bar.log, /foo/1/2/bar.log, etc.

And will not match /foo/bar.log, or /foo/3/notbar.log

Note: Because a single ellipse recurses through all directories and subdirectories, /foo/.../bar.log matches the same as /foo/.../.../bar.log.

* The asterisk wildcard matches anything in that specific directory path segment.

Unlike "...", "*" doesn't recurse through any subdirectories.

[^/]* /foo/*/bar matches the files /foo/bar, /foo/1/bar, /foo/2/bar, etc. However, it does not match /foo/1/2/bar.

/foo/m*r/bar matches /foo/mr/bar, /foo/mir/bar, /foo/moor/bar, etc.

/foo/*.log matches all files with the .log extension, such as /foo/bar.log. It does not match /foo/bar.txt or /foo/bar/test.log.

Note: A single dot (.) is not a wildcard, and is the regex equivalent of \..

For more specific matches, combine the ... and * wildcards. For example, /foo/.../bar/* matches any file in the /bar directory within the specified path.

Caution: In Windows, you cannot currently use a wildcard at the root level. For example, this does not work:

[monitor://E:\...\foo\*.log]

Splunk Enterprise logs an error and fails to index the desired files.

This is a known issue, described in the Known Issues topic of the Release Notes. Look there for details on all known issues.

Wildcards and regular expression metacharacters

When determining the set of files or directories to monitor, Splunk Enterprise splits elements of a monitoring stanza into segments - defined as text between directory separator characters ("/" or "\") in the stanza definition. If you specify a monitor stanza that contains segments with both wildcards and regex metacharacters (such as (, ), [, ], and |), those characters behave differently depending on where the wild card is in the stanza.

If a monitoring stanza contains a segment with regex metacharacters before a segment with wildcards, Splunk Enterprise treats the metacharacters literally, as if you wanted to monitor files or directories with those characters in the files' or directories' names. For example:

[monitor:///var/log/log(a|b).log]

monitors the /var/log/log(a|b).log file. Splunk Enterprise does not treat the (a|b) as a regular expression because there are no wildcards present.

[monitor:///var/log()/log*.log]

monitors all files in the /var/log()/ directory that begin with log and have the extension .log. Splunk Enterprise does not treat the () as a regular expression because the regex is in the segment before the wildcard.

If the regex metacharacters occur within or after a segment that contains a wildcard, Splunk Enterprise treats the metacharacters as a regex and matches files to monitor accordingly. For example:

[monitor:///var/log()/log(a|b)*.log]

monitors all files in the /var/log()/ directory that begin with either loga or logb and have the extension .log. Splunk does not treat the first set of () as a regex because the wild card is in the following segment. The second set of () gets treated as a regex because it is in the same segment as the wildcard '*'.

[monitor:///var/.../log(a|b).log]

monitors all files in any subdirectory of the /var/ directory named loga.log and logb.log. Splunk treats (a|b) as a regex because of the wildcard '...' in the previous stanza segment.

[monitor:///var/.../log[A-Z0-9]*.log]

monitors all files in any subdirectory of the /var/ directory that:

  • begin with log, then
  • contain a single capital letter (from A-Z) or number (from 0-9), then
  • contain any other characters, then
  • end in .log.

Splunk Enterprise treats [A-Z0-9]* as a regex because of the wildcard '...' in the previous stanza segment.

Input examples

To monitor /apache/foo/logs, /apache/bar/logs, /apache/bar/1/logs, etc.:

[monitor:///apache/.../logs/*]

To monitor /apache/foo/logs, /apache/bar/logs, etc., but not /apache/bar/1/logs or /apache/bar/2/logs:

[monitor:///apache/*/logs]

To monitor any file directly under /apache/ that ends in .log:

[monitor:///apache/*.log]

To monitor any file under /apache/ (under any level of subdirectory) that ends in .log:

[monitor:///apache/.../*.log]

The "..." followed by a folder separator will imply that the wildcard level folder will be excluded.

[monitor:///var/log/.../*.log]

the tailing logic will become '^\/var\/log/.*/[^/]*\.log$'

Therefore /var/log/subfolder/test.log will match, but /var/log/test.log will not match and be skipped. to monitor all files in all folders use

[monitor:///var/log/]

whitelist=\.log$

recurse=true

#true by default

Wildcards and whitelisting

Important: Splunk Enterprise defines whitelists and blacklists with standard Perl-compatible Regular Expression (PCRE) syntax, unlike the file input path syntax described in the previous sections.

When you specify wildcards in a file input path, Splunk Enterprise creates an implicit whitelist for that stanza. The longest wildcard-free path becomes the monitor stanza, and Splunk Enterprise translates the wildcards into regular expressions, as listed in the table above.

Additionally, Splunk Enterprise anchors the converted expression to the right end of the file path, so that the entire path must be matched.

For example, if you specify

[monitor:///foo/bar*.log]

Splunk Enterprise translates this into

[monitor:///foo/]
whitelist = bar[^/]*\.log$

On Windows, if you specify

[monitor://C:\Windows\foo\bar*.log]

Splunk Enterprise translates it into

[monitor://C:\Windows\foo\]
whitelist = bar[^/]*\.log$

Note: In Windows, whitelist and blacklist rules do not support regexes that include backslashes; you must use two backslashes \\ to escape wildcards.

For more information on using whitelists with file inputs, see "Whitelist or blacklist specific incoming data".

PREVIOUS
Edit inputs.conf
  NEXT
Whitelist or blacklist specific incoming data

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14


Comments

wild card searching on windows appears to be case sensitive. I.e. from the previous example monitor://c:\Program Files\foo\*.log and monitor://c:\Program Files\foo\*.LOG appear to be case sensitive.

Cneberg
January 6, 2014

It would be good to have more detailed windows examples. I have c:\dir1\dir2\dir3\dir4\filename.txt I would like to index only the txt files in the dir4 directory <br />[monitor://c:\dir1\…]<br />sourcetype = Test<br />whitelist = *\dir4\.txt$<br />This seems to take all files where i only want dir4 text files.<br /><br />Any ideas?

Andykiely
October 4, 2012

Regading vbumgarner's comment.<br /><br />Both of the first two inputs will monitor either a file or a directory called foo, it simply depends upon what is actually on the disk when splunk starts up.<br /><br />However the second stanza clearly shows the intent to monitor a directory, so perhaps it is useful in terms of a self-documenting configuration.

Jrodman
September 7, 2011

More Windows examples:<br /><br />[monitor://c:\Program Files\foo]<br />will look for a FILE called foo.<br /><br />[monitor://c:\Program Files\foo\]<br />will look for all files in the directory foo.<br /><br />[monitor://c:\Program Files\foo\*.log]<br />will look for files matching *.log in the directory foo.<br /><br />[monitor://c:\Program Files\foo\*.log]<br />will look for files matching *.log in the directory foo.

Vbumgarner
July 29, 2011

we need examples for monitoring windows directories; [monitor://c:\windows\system32\logfiles\w3svc1]

Pstraw
June 22, 2011

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters