Splunk® Enterprise

Getting Data In

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Override source types on a per-event basis

This topic shows you how to configure Splunk to override source types on a per-event basis. You do this at parse-time, after Splunk has made its initial assignment as described in "How Splunk assigns source types".

To configure per-event overrides, you use transforms.conf in tandem with props.conf.

Note: Since this type of override occurs at parse-time, it works only on an indexer or heavy forwarder, not on a universal forwarder. See "Configuration parameters and the data pipeline" in the Admin manual for more information on what configurations are available at different points in the input/parsing/indexing process.

For information about configuring basic (not per-event) source type overrides for event data that comes from specific inputs or that has a particular source, see "Override automatic source type assignment" in this manual.

Configuration

To configure per-event overrides, you need to create two stanzas, one in transforms.conf and another in props.conf. Edit these files in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information about configuration files in general, see "About configuration files" in the Admin manual.

transforms.conf

Create a stanza in transforms.conf that follows this syntax:

[<unique_stanza_name>]
REGEX = <your_regex>
FORMAT = sourcetype::<your_custom_sourcetype_value>
DEST_KEY = MetaData:Sourcetype

Note the following:

  • <unique_stanza_name> should reflect that it involves a source type. You'll use this name later in the props.conf stanza.
  • <your_regex> is a regular expression that identifies the events that you want to apply a custom source type to (such as events carrying a particular hostname or other field value).
  • <your_custom_sourcetype_value> is the source type that you want to apply to the regex-selected events.

Note: For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test regexes by using them in searches with the rex search command. Splunk also maintains a list of useful third-party tools for writing and testing regular expressions.

props.conf

Next, create a stanza in props.conf that references the transforms.conf stanza:

[<spec>]
TRANSFORMS-<class> = <unique_stanza_name>

Note the following:

  • <spec> can be:
    • <sourcetype>, the source type of an event.
    • host::<host>, where <host> is the host value for an event.
    • source::<source>, where <source> is the source value for an event.
  • <class> is any unique identifier that you want to give to your transform.
  • <unique_stanza_name> is the name of the stanza you created in transforms.conf.

Example: Assign a source type to events from a single input but different hosts

Let's say that you have a shared UDP input, UDP514. Your Splunk instance indexes a wide range of data from a number of hosts through this input. You've found that you need to apply a particular source type called "my_log" to data originating from three specific hosts (host1, host2, and host3) reaching Splunk through UDP514.

To start, you can use the regex that Splunk typically uses to extract the host field for syslog events. You can find it in system/default/transforms.conf:

[syslog-host]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1
DEST_KEY = MetaData:Host

You can easily modify this regex to only match events from the hostnames you want (in this example, host1, host2, and host3):

REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(host1|host2|host3)[\w\.\-]*\]?\s

Now you can use the modified regex in a transform that applies the my_log source type to events that come from those three hosts:

[set_sourcetype_my_log_for_some_hosts]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(host1|host2|host3)[\w\.\-]*\]?\s
FORMAT = sourcetype::my_log
DEST_KEY = MetaData:Sourcetype

Then you can specify that transform in a props.conf stanza that identifies the specific input for the events:

[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_my_log_for_some_hosts
PREVIOUS
List of pretrained source types
  NEXT
Create source types

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters