Override source types on a per-event basis

You can override source types on a per-event basis on the Splunk platform by using a heavy forwarder or Edge Processor to assign the events to a new source type and sending those events to Splunk Cloud Platform. On Splunk Enterprise, you can override source types directly on the instance itself.

This source type assignment happens at parse-time, after the platform has made its initial source type assignment. For more information about this process, see How the Splunk platform assigns source types in Why source types matter.

Since this type of override occurs at parse-time, the override works only on an indexer, heavy forwarder or Edge Processor. It doesn't work on a universal forwarder or directly on Splunk Cloud Platform. See Configuration parameters and the data pipeline in the Admin Manual for more information on what configurations are available at different points in the input, parsing, and indexing processes.

To configure per-event overrides, use the transforms.conf and props.conf configuration files in tandem to specify the events that must use a new source type, along with the source type that the events must use.

For information about configuring basic source type overrides for event data that comes from specific inputs or that has a particular source, see Override automatic source type assignment.

Configuration

To configure per-event overrides, create one stanza in the transforms.conf file and another in the props.conf file. Edit these files in the $SPLUNK_HOME/etc/system/local/ directory or in your own custom application directory at $SPLUNK_HOME/etc/apps/. For more information about configuration files in general, see About configuration files in the Admin Manual.

Edit the transforms.conf file

1. Open $SPLUNK_HOME/etc/system/local/transforms.conf file for editing.
2. Create a stanza in transforms.conf that follows this syntax:

   [<unique_stanza_name>]
   REGEX = <your_regex>
   FORMAT = sourcetype::<your_custom_sourcetype_value>
   DEST_KEY = MetaData:Sourcetype
   

3. Save the file and close it.

In the file, the settings have the following meaning:

• <unique_stanza_name> means that it involves a source type. You'll use this name later in the props.conf stanza.
• <your_regex> is a regular expression that identifies the events that you want to apply a custom source type to, such as events that carry a particular host name or other field value.
• <your_custom_sourcetype_value> is the source type that you want to apply to the events that <your_regex> selected.

You can test regular expressions by using them in searches with the rex search command. See rex in the Search Reference.

Edit the props.conf file

1. Open $SPLUNK_HOME/etc/system/local/props.conf.
2. Create a stanza in the props.conf file that references the stanza that you specified in the transforms.conf file:

   [<spec>]
   TRANSFORMS-<class> = <unique_stanza_name>
   

   Refer to the following table for the meanings of each placeholder variable within this stanza:

   Placeholder variable	Description
   spec	Can be set to the following options: <sourcetype>, or the source type of an event host::<host>, where <host> is the host value for an event source::<source>, where <source> is the source value for an event
   <class>	Any unique identifier that you want to give to your transform
   <unique_stanza_name>	The name of the stanza you created in transforms.conf

3. Save the file and close it.
4. Restart the Splunk platform instance.

Example: Assign a source type to events from a single input but different hosts

Suppose that you have a shared UDP input, UDP514. Your Splunk platform instance indexes a wide range of data from a number of hosts through this input. You find that you need to apply a particular source type called my_log to data originating from three specific hosts, host1, host2, and host3, reaching your instance through the UDP514 input.

To start, you can use the regular expression that Splunk software typically uses to extract the host field for syslog events. You can find it in $SPLUNK_HOME/etc/system/default/transforms.conf:

[syslog-host]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1
DEST_KEY = MetaData:Host

You can modify this regular expression to match events from only the host names you want. In this example, the host names are host1, host2, and host3:

REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(host1|host2|host3)[\w\.\-]*\]?\s

Now you can use the modified regular expression in a transform that applies the my_log source type to events that come from those three hosts:

[set_sourcetype_my_log_for_some_hosts]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(host1|host2|host3)[\w\.\-]*\]?\s
FORMAT = sourcetype::my_log
DEST_KEY = MetaData:Sourcetype

Then you can specify that transform in a props.conf stanza that identifies the specific input for the events:

[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_my_log_for_some_hosts