Documentation > Splunk > Developing Dashboards, Views, and Apps for Splunk Web > Create Regular Expressions

Developing Dashboards, Views, and Apps for Splunk Web

 


Start Here
Command Line Interface
SOAP API
Bundles
Modules
Skins
Workflow Integration
Search Language
Built-In Processors

Create Regular Expressions

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Create Regular Expressions

These parameters let you create regular expressions that can be invoked by properties to match patterns in data and meta data.


Splunk uses the PCRE library for Perl Compatible Regular Expressions. For additional information on building your own regular expressions, see the perlre documentation as well as the PCRE man pages for differences between PCRE and Perl 5. (Search for "PCRECOMPAT(3)".)


Filename

Format 

[<regex name>]
attribute1 = val1
attribute2 = val2
...

Attributes

If the same <spec> is found in two bundle directories, the following precedence rules apply:


Overriding is performed attribute by attribute.


SOURCE_KEY = <string> (_raw) : Specifies which key in the event should be the source of the string on which to perform the regex.


LOOKAHEAD = <integer> (256) : Specifies how far into the string to match.


REGEX = <regular expression> (Empty) : If set, this is the regular expression that will operate on the data.


FORMAT = <string> ($1) : A string with arguments specified by $n, where n represents the output of the n-th parenthesized grop of the REGEX. If the regex does not have n groups, the matching fails. The special identifier $0 represents what was in the DEST_KEY before this regex was performed.


DEFAULT_VALUE = <string> (Empty) : If set, and the regex fails, this value will be written to the DEST_KEY.


DEST_KEY = <string> (<regex name>) : Specifies where the formatted result of running the regex should be stored. Any key prefixed by '_' will not be stored when it reaches the index.


REPEAT_MATCH = <bool> (False) : Specifies whether the regular expression should be run several times on the SOURCE_KEY. If set, it will start wherever the last match stopped, and will continue until no more matches are found.


WRITE_META = <string> (False) : Specifies that whatever was written to DEST_KEY should also be appended to the "meta data" for the event.


Splunk Reserved Keys

As data is streamed through Splunk pipelines, it lives in a set of keys. Some keys are indexed and/or stored and others are neither.


You can create new keys, any you create without _ prefixes are indexed in entirety (no major/minor segmentation) but not stored. Any key prefixed by '_' will not be stored when it reaches the index.


_raw : The raw text of the event. It is indexed with major/minor segmentation. It is intended to preserve the integrity of original content. You can change _raw with regexes, but you will destroy data integrity.


_done : If set to any string this is the last event in a stream.


_meta : The metadata for the event. It is indexed with space delimited segmentation.


_time : The timestamp of the event, in seconds since 1/1/1970 UTC. It is in the primary index but not stored.


queue : The queue to be used by the next queue output processor. It is neither indexed nor stored.


cluster : Used to create meta-events. It is indexed in entirety but not stored.


MetaData:FinalType : The event type hash. It is indexed in entirety but not stored.


MetaData:Host : The host associated with the event. The value must be prefixed by "host::" It is indexed in entirety but not stored.


_MetaData:Index : The index where the event should be stored. It is neither indexed nor stored.


_MetaData:IndexTerms : Used by Splunk to capture terms to be indexed but not stored. It is indexed with major/minor segmentation.


MetaData:Source : The source associated with the event. The value must be prefixed by "source::" It is indexed in entirety but not stored.


MetaData:Sourcetype : The sourcetype of the event. The value must be prefixed by "sourcetype::" It is indexed in entirety but not stored.


Examples

This creates a meta event for each IP address via regex. 


[access-ip]
DEST_KEY      = cluster
REGEX         = (\d+\.\d+\.\d+\.\d+)
DEFAULT_VALUE = 
PREPEND       = ip::
WRITE_META    = False

To make use of the meta events, you can Splunk for anything and add the modifer index::metaevents. Or, if you search for two values that are in separate events but in the same meta event (because both events contain the same IP value), the meta event will appear in your results as if it were a real log event.

Retrieved from "http://docs.splunk.com/Documentation/Splunk/2.2/Developer/CreateRegularExpressions"

This documentation applies to the following versions of Splunk: 2.1 , 2.2 , 2.2.1 , 2.2.3 , 2.2.6 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.