Extract and add new fields
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Extract and add new fields
As you learn more about your data, you may find more information that you want to use. There are a variety of ways to extract this information from your events, save it as a field, and use it to search and build reports. You can also look up information from external sources (such as a CSV file or the output of a script) and add it to your event data.
In this topic, you will:
- Learn how to extract and save fields interactively in Splunk Web.
- Learn about the search commands that extract fields from your events.
- Learn about using configuration files to define field extraction at index-time.
- Learn about matching fields with lookup tables to add new fields to your events.
Extract fields interactively in Splunk Web
You can create custom fields dynamically using the interactive field extraction (IFX) feature of Splunk Web. To access the IFX, run a search and then select "Extract fields" from the dropdown that appears left of the timestamps in the search results. The IFX enables you to extract one field at a time, based on a host, source, or source-type value. For more information, see the Interactive field extraction example in this manual.
Extract fields with search commands
You can use a variety of search commands to extract fields in different ways. Here is a list of those commands; for examples of how to use each of these commands, see "Extract fields with search commands" in this manual.
- rex performs field extractions using Perl regular expressions named groups.
- extract (or
kv, for "key/value") explicitly extracts field/values using default patterns.
- multikv extracts field/values on multi-line, tabular-formatted events.
- xmlkv extracts field/values on xml-formatted event data.
- kvform extracts field/values based on predefined form templates.
Define field extraction in conf files
All field extraction rules that you add using IFX get written to the configuration files. You can also edit these files directly, if you have the permissions to access them. For more information see "Add fields at search time through configuration file edits" in the Knowledge Manager manual.
Look up fields from external data sources
You can match fields in your events to fields in external sources, such as lookup tables, and use these matches to add more information inline to your events.
A lookup table can be a static CSV file or the output of a Python script. You can also use the results of a search to populate the CSV file and then set that up as a lookup table. For more information about field lookups, see "Add fields from external data sources" in the Knowledge Manager manual.
After you configure a fields lookup, you can invoke it from the Search app with the
Example: Given a field lookup named
dnslookup, referencing a Python script that performs a DNS and reverse DNS lookup and accepts either a host name or IP address as arguments -- you can use the lookup command to match the host name values in your events to the host name values in the table, and then add the corresponding IP address values to your events.
... | lookup dnslookup host OUTPUT ip
For a more extensive example using the Splunk script
external_lookup.py, see "Reverse DNS Lookups for Host Entries" in the Splunk blogs.
This documentation applies to the following versions of Splunk: 4.1 , 4.1.1 , 4.1.2 , 4.1.3 , 4.1.4 , 4.1.5 , 4.1.6 , 4.1.7 , 4.1.8 , 4.2 , 4.2.1 , 4.2.2 , 4.2.3 , 4.2.4 , 4.2.5 , 4.3 , 4.3.1 , 4.3.2 , 4.3.3 , 4.3.4 , 4.3.5 , 4.3.6