transforms.conf

transforms.conf

Transforms.conf specifies transformations to apply to events based on regex-based patterns, including rules for extracting fields or masking event text.

These transformations are applied to events from particular sources, hosts or sourcetypes via props.conf.

transforms.conf.spec

# Copyright (C) 2005-2008 Splunk Inc.  All Rights Reserved.  Version 3.0
#
# This file contains possible attributes and values you can use to configure transform
# and event signing in transforms.conf.
#
# There is a transforms.conf in $SPLUNK_HOME/etc/system/default/.  To set custom configurations, 
# place a transforms.conf $SPLUNK_HOME/etc/system/local/.  For examples, see transforms.conf.example.
# You can enable configurations changes made to transforms.conf by typing the following search string
# in Splunk Web:
#
# | extract reload=T 
#
# To learn more about configuration files (including precedence) please see the documentation 
# located at http://www.splunk.com/base/Documentation/latest/Admin/HowDoConfigurationFilesWork.
[<unique_stanza_name>]
* Name your stanza. Use this name when configuring props.conf.
  For example, in a props.conf stanza, enter TRANSFORMS-<value> = <unique_stanza_name>.
* Follow this stanza name with any number of the following attribute/value pairs.  
* If you do not specify an entry for each attribute, Splunk uses the default value.
REGEX = <regular expression>
* Enter a regular expression to operate on the data.
* Defaults to empty.
LOOKAHEAD = <integer>
* Specify how many characters to search into an event.
* Defaults to 256.
DEST_KEY = <KEY>
* Specify where to store the results of the REGEX.
* Use the KEYs listed below.
FORMAT = <string>
* Specify the format of the event, including any fields names or values you want to add.
* Use $n (e.g. $1, $2, etc) to specify the output of each REGEX match. 
* If the regex does not have n groups, the matching fails. 
* The special identifier $0 represents what was in the DEST_KEY before this regex was performed.
* Defaults to $1.
WRITE_META = <true | false>
* Automatically writes REGEX to metadata.
* Use instead of DEST_KEY = meta.
* Defaults to false.
DEFAULT_VALUE = <string>
* If set, and REGEX (above) fails, write this value to DEST_KEY.
* Defaults to empty.
SOURCE_KEY = <string>
* Set which KEY to perform the regex on.
* Use the KEYs listed below.
* Defaults to _raw (the raw event).
REPEAT_MATCH = <true | false>
* Specify whether to run REGEX several times on the SOURCE_KEY. 
* REPEAT_MATCH starts wherever the last match stopped, and continues until no more matches are found.
* Defaults to false.
DELIMS = <quoted string>
* Set delimiter characters to separate data into key-value pairs, and then to separate key from value.
* NOTE: Delimiters must be quoted with " " (to escape, use \).
* Usually, two sets of delimiter characters must be specified: 
  The first to extract key/value pairs.
  The second to separate the key from the value.
* If you enter only one set of delimiter characters, then the extracted tokens:
  Are named with names from FIELDS, if FIELDS are entered (below).
  OR even tokens are used as field names while odd tokens become field values. 
* Consecutive delimiter characters are consumed except when a list of field names is specified. 
  
FIELDS = <quoted string list>
* List the names of the field values extracted using DELIMS.
* NOTE: If field names contain spaces or commas they must be quoted with " " (to escape, use \).
* Defaults to "".
#######
# KEYS:
#######
* NOTE: Keys are case-sensitive.  Use the following keys exactly as they appear.
_raw  : The raw text of the event.
_done : If set to any string, this is the last event in a stream.
_meta : A space separated list of metadata for an event.
_time : The timestamp of the event, in seconds since 1/1/1970 UTC.
MetaData:FinalType  : The event type of the event.
MetaData:Host       : The host associated with the event.
                      The value must be prefixed by "host::"
_MetaData:Index     : The index where the event should be stored.
MetaData:Source     : The source associated with the event.
                      The value must be prefixed by "source::"
MetaData:Sourcetype : The sourcetype of the event.
                      The value must be prefixed by "sourcetype::"
* NOTE: Any KEY prefixed by '_' is not indexed by Splunk, in general.

transforms.conf.example

# Copyright (C) 2005-2008 Splunk Inc.  All Rights Reserved.  Version 3.0 
#
# This is an example transforms.conf.  Use this file to create regexes and rules for transforms.
# Use this file in tandem with props.conf.
#
# To use one or more of these configurations, copy the configuration block into transforms.conf 
# in $SPLUNK_HOME/etc/system/local/. You must restart Splunk to enable configurations.
#
# To learn more about configuration files (including precedence) please see the documentation 
# located at http://www.splunk.com/base/Documentation/latest/Admin/HowDoConfigurationFilesWork.
# Note: These are examples.  Replace the values with your own customizations.
# Indexed field:
[netscreen-error]
REGEX =  device_id=[^ ]+\s+\[w+\](.*)(?
FORMAT = err_code::$1
WRITE_META = true
# Extracted field:
[netscreen-error]
REGEX = device_id=[^ ]+\s+\[w+\](.*)(?
FORMAT = err_code::$1
# Override host:
[hostoverride]
DEST_KEY = MetaData:Host
REGEX = \s(\w*)$
FORMAT = host::$1
# Extracted fields:
[netscreen-error]
REGEX = device_id=[^ ]+\s+\[w+\](.*)(?
FORMAT = err_code::$1
# Mask sensitive data:
[session-anonymizer]
REGEX = (?m)^(.*)SessionId=\w+(\w{4}[&"].*)$
FORMAT = $1SessionId=########$2
DEST_KEY = _raw
# Route to an alternate index:
[AppRedirect]
REGEX = Application
DEST_KEY = _MetaData:Index
FORMAT = Verbose
# Extract comma-delimited values into fields:
[extract_csv]
DELIMS = ","
FIELDS = "field1", "field2", "field3"
# This example assigns the extracted values from _raw to field1, field2 and field3 (in order of 
# extraction). If more than three values are extracted the values without a matching field name 
# are ignored.
# Extract key-value pairs
# This example extracts key-value pairs which are separated by '|'
# while the key is delimited from value by '='.
[pipe_eq]
DELIMS = "|", "="
# This example extracts key-value pairs which are separated by '|'
# while the key is delimited from value by '='.

• Was this topic useful?
• Post a comment