Extract fields with search commands

You can use search commands to extract fields in different ways.

• The rex command performs field extractions using named groups in Perl regular expressions.
• The extract (or kv, for key/value) command explicitly extracts field and value pairs using default patterns.
• The multikv command extracts field and value pairs on multiline, tabular-formatted events.
• The spath command extracts field and value pairs on structured event data, such as XML and JSON.
• The xmlkv and xpath commands extract field and value pairs on XML-formatted event data.
• The kvform command extracts field and value pairs based on predefined form templates.

In Splunk Web, you can define field extractions on the Settings > Fields > Field Extractions page.

The following sections describe how to extract fields using regular expressions and commands. See About fields in the Knowledge Manager Manual.

Extract fields using regular expressions

The rex command performs field extractions using named groups in Perl regular expressions that you include in the search criteria. The rex command matches segments of your raw events with the regular expression and saves these matched values into a field.

In this example, values that occur after the strings From: and To: are saved into the from and to fields.

... | rex field=_raw "From: (?<from>.*) To: (?<to>.*)"

If a raw event contains From: Susan To: Bob, the search extracts the field name and value pairs: from=Susan and to=Bob.

For a primer on regular expression syntax and usage, see www.regular-expressions.info. The following are useful third-party tools for writing and testing regular expressions:

• regex101
• RegExr
• Debuggex

Extract fields from .conf files

The extract command forces field/value extraction on the result set. If you use the extract command without specifying any arguments, fields are extracted using field extraction stanzas that have been added to the props.conf file. You can also use the extract command to test field extractions that you add to the conf files.

Extract fields from events formatted as tables

Use the multikv command to force field and value extractions on multiline, tabular-formatted events. The multikv command creates a new event for each table row and derives field names from the table title.

Extract fields from events formatted in XML

The xmlkv command enables you to force field and value extractions on XML-formatted tags in event data, such as transactions from web pages.

Extract fields from XML and JSON documents

The spath command extracts information from structured data formats, such as XML and JSON, and store the extracted values in fields.

Extract fields from events based on form templates

The kvform command extracts field and value pairs from events based on form templates that are predefined and stored in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For example, if form=sales_order, the search looks for a sales_order.form, and matches all processed events against that form to extract values.

For Splunk Cloud Platform, you must create a private app to extract fields using form templates. If you are a Splunk Cloud administrator with experience creating private apps, see Manage private apps in your Splunk Cloud deployment in the Splunk Cloud Admin Manual. If you have not created private apps, contact your Splunk account representative for help with this customization.