Splunk® Enterprise

User Manual

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Extract and add new fields

As you learn more about your data, you may find more information that you want to use. There are a variety of ways to extract this information from your events, save it as a field, and use it to search and build reports. You can also look up information from external sources (such as a CSV file or the output of a script) and add it to your event data.

In this topic, you will:

  • Learn how to extract and save fields interactively in Splunk Web.
  • Learn about the search commands that extract fields from your events.
  • Learn about using configuration files to define field extraction at index-time.
  • Learn about matching fields with lookup tables to add new fields to your events.

Extract fields interactively in Splunk Web

You can create custom fields dynamically using the interactive field extraction (IFX) feature of Splunk Web. To access the IFX, run a search and then select "Extract fields" from the dropdown that appears left of the timestamps in the search results. The IFX enables you to extract one field at a time, based on a host, source, or source-type value. For more information, see the Interactive field extraction example in this manual.

Extract fields with search commands

You can use a variety of search commands to extract fields in different ways. Here is a list of those commands; for examples of how to use each of these commands, see "Extract fields with search commands" in this manual.

  • rex performs field extractions using Perl regular expressions named groups.
  • extract (or kv, for "key/value") explicitly extracts field/values using default patterns.
  • multikv extracts field/values on multi-line, tabular-formatted events.
  • xmlkv extracts field/values on xml-formatted event data.
  • kvform extracts field/values based on predefined form templates.

Define field extraction in conf files

All field extraction rules that you add using IFX get written to the configuration files. You can also edit these files directly, if you have the permissions to access them. For more information see "Add fields at search time through configuration file edits" in the Knowledge Manager manual.

Look up fields from external data sources

You can match fields in your events to fields in external sources, such as lookup tables, and use these matches to add more information inline to your events.

A lookup table can be a static CSV file or the output of a Python script. You can also use the results of a search to populate the CSV file and then set that up as a lookup table. For more information about field lookups, see "Add fields from external data sources" in the Knowledge Manager manual.

After you configure a fields lookup, you can invoke it from the Search app with the lookup command.

Example: Given a field lookup named dnslookup, referencing a Python script that performs a DNS and reverse DNS lookup and accepts either a host name or IP address as arguments -- you can use the lookup command to match the host name values in your events to the host name values in the table, and then add the corresponding IP address values to your events.

... | lookup dnslookup host OUTPUT ip

For a more extensive example using the Splunk script external_lookup.py, see "Reverse DNS Lookups for Host Entries" in the Splunk blogs.

Tag and alias field values
Extract fields with search commands

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters