Assign metadata to events dynamically

Assign metadata to events dynamically

Ths feature allows you to dynamically assign metadata to files as they are being consumed by Splunk. Use this feature to specify source type, host, or other metadata dynamically for for incoming data. This feature is useful mainly with scripted data -- either a scripted input or a pre-existing file processed by a script.

Important: Splunk does not recommend using dynamic metadata assignment with ongoing monitoring (tail) inputs. For more information about file inputs, refer to Monitor files and directories in this manual.

To use this feature, you append a single dynamic input header to your file and specify the metadata fields you want to assign values to. The metadata fields most likely to be of interest are sourcetype, host, and source. You can see the list of all available pipeline metadata fields in transforms.conf.spec.

You can use this method to assign metadata instead of editing inputs.conf, props.conf and transforms.conf.

Configure a single input file

To use this feature for an existing input file, edit the file (either manually or with a script) to add a single input header:

      ***SPLUNK*** <metadata field>=<string> <metadata field>=<string> ...

• Set <metadata field>=<string> to a valid metadata/value pair. You can specify mutiple pairs. For example, sourcetype=log4j host=swan.
• Add the single header anywhere in your file. Any data following the header will be appended with the attributes and values you assign until the end of the file is reached.
• Add your file to $SPLUNK_HOME/var/spool/splunk or any other directory being monitored by Splunk.

Configure with a script

In the more common scenario, you write a script to dynamically add an input header to your incoming data stream. Your script can also set the header dynamically based on the contents of the input file.

• Was this topic useful?
• Post a comment