Extract fields with search commands
You can use search commands to extract fields in different ways.
rexcommand performs field extractions using named groups in Perl regular expressions.
kv, for key/value) command explicitly extracts field and value pairs using default patterns.
multikvcommand extracts field and value pairs on multiline, tabular-formatted events.
spathcommand extracts field and value pairs on structured event data, such as XML and JSON.
xpathcommands extract field and value pairs on XML-formatted event data.
kvformcommand extracts field and value pairs based on predefined form templates.
In Splunk Web, you can define field extractions on the Settings > Fields > Field Extractions page.
The following sections describe how to extract fields using regular expressions and commands. See About fields in the Knowledge Manager Manual.
Extract fields using regular expressions
rex command performs field extractions using named groups in Perl regular expressions that you include in the search criteria. The
rex command matches segments of your raw events with the regular expression and saves these matched values into a field.
In this example, values that occur after the strings
To: are saved into the from and to fields.
... | rex field=_raw "From: (?<from>.*) To: (?<to>.*)"
If a raw event contains
From: Susan To: Bob, the search extracts the field name and value pairs: from=Susan and to=Bob.
For a primer on regular expression syntax and usage, see www.regular-expressions.info. The following are useful third-party tools for writing and testing regular expressions:
Extract fields from .conf files
The extract command forces field/value extraction on the result set. If you use the
extract command without specifying any arguments, fields are extracted using field extraction stanzas that have been added to the
props.conf file. You can also use the
extract command to test field extractions that you add to the conf files.
Extract fields from events formatted as tables
multikv command to force field and value extractions on multiline, tabular-formatted events. The
multikv command creates a new event for each table row and derives field names from the table title.
Extract fields from events formatted in XML
xmlkv command enables you to force field and value extractions on XML-formatted tags in event data, such as transactions from web pages.
Extract fields from XML and JSON documents
spath command extracts information from structured data formats, such as XML and JSON, and store the extracted values in fields.
Extract fields from events based on form templates
kvform command extracts field and value pairs from events based on form templates that are predefined and stored in
$SPLUNK_HOME/etc/system/local/, or your own custom application directory in
$SPLUNK_HOME/etc/apps/. For example, if
form=sales_order, the search looks for a
sales_order.form, and matches all processed events against that form to extract values.
If you have Splunk Cloud and want to use form templates for field extraction, file a Support ticket.
Use lookup to add fields from lookup tables
Evaluate and manipulate fields with multiple values
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 6.3.1, 6.6.9, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 7.0.10, 7.0.11, 7.0.13