Use the Field extractions page

Use the Field extractions page in Settings to manage search-time field extractions. There are three methods by which you can add search-time field extractions. You can:

• Use the field extractor to create extractions. This method is relatively easy and does not require you to understand how regular expressions work.
• Make direct edits to props.conf. You need Splunk Enterprise to use this method.
• Add new field extractions with the Field extractions page.

The Field extractions page enables you to:

• Review the overall set of search-time extractions that you have created or which your permissions enable you to see, for all Apps in your Splunk deployment.
• Create new search-time field extractions.
• Change permissions for field extractions. Field extractions created through the field extractor and the Field extractions page are initially only available to their creators until they are shared with others.
• Delete field extractions, if your app-level permissions enable you to do so, and if they are not default extractions that were delivered with the product. Default knowledge objects cannot be deleted. For more information about deleting knowledge objects, see Disable or delete knowledge objects.

If you have additional write permissions for a particular search-time field extraction, the Field extractions page also enables you to:

• Update its regular expression, if it is an inline field extraction..
• Add or delete named extractions that have been defined in transforms.conf or the Field transactions page in Splunk Web, if it uses transactions.

Note: You cannot manage index-time field extractions in Splunk Web. We do not recommend that you change your set of index-time field extractions, but if you need to, you have to modify your props.conf and transforms.conf configuration files manually. For more information about index-time field extraction configuration, see "Configure index-time field extractions" in the Getting Data In Manual.

Navigate to the Field extractions page by selecting Settings > Fields > Field extractions.

Review search-time field extractions in Splunk Web

To better understand how the Field extractions page displays your field extraction, it helps to understand how field extractions are set up in your props.conf and transforms.conf files.

Field extractions can be set up entirely in props.conf, in which case they are identified on the Field extractions page as inline field extractions. Some field extractions include a transforms.conf component, and these types of field extractions are called transform field extractions. To create or edit that component of the field extraction via Splunk Web, use the Field Transforms page in Splunk Web.

For more information about transforms and the Field Transforms page, see use the field transformations page.

For more information about field extraction setup directly in the props.conf and transforms.conf files see Create and maintain search-time field extractions through configuration files.

Name column

The Name column in the Field extractions page displays the overall name (or "class") of the field extraction. The field extraction format is:

<spec> : [EXTRACT-<class> | REPORT-<class>]

• <spec> can be:
  • <sourcetype>, the source type of an event.
  • host::<host>, where <host> is the host for an event.
  • source::<source>, where <source> is the source for an event.

EXTRACT-<class> field extractions are extractions that are only defined in props.conf. They are created automatically by field extractions made through IFX and certain search commands. If you have Splunk Enterprise, you can also add them by making direct updates to the props.conf file. This kind of extraction is always associated with a field-extracting regular expression. On the Field extractions page, this regex appears in the Extraction/Transform column.

REPORT-<class> field extractions reference field transform stanzas in transforms.conf. This is where their field-extracting regular expressions are located. On the Field extractions page, the referenced field transform stanza is indicated in the Extraction/Transform column.

You can work with transforms in Splunk Web through the Field Transformations page. See Use the Field Transformations page in Splunk Web.

Type column

There are two field extraction types: inline and transforms.conf.

• Inline extractions always have EXTRACT-<class> configurations. They are entirely defined within props.conf; they do not reference external field transforms.
• Uses transform extractions always have REPORT-<class> name configurations. As such they reference field transforms in transforms.conf. You can define field transforms directly in transforms.conf or via Splunk Web using the Field transformations page.

Extraction Transform column

In the Extraction/Transform column, Splunk Web displays different things depending on the field extraction Type.

• For inline extraction types, Splunk Web displays the regular expression that Splunk software uses to extract the field. The named group (or groups) within the regex show you what field(s) it extracts.

You can use regular expressions with inline field extractions to apply your inline field extraction to several sourcetypes. For example, you could have multiple sourcetypes named foo_apache_access, bar_apache_access, baz_apache_access, quux_apache-access. You can apply your field extraction to these sourcetypes by using the following as your sourcetype: (?::){0}*_apache_access

For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command.

• In the case of Uses transform extraction types, Splunk Web displays the name of the transforms.conf field transform stanza (or stanzas) that the field extraction is linked to through props.conf. A field extraction can reference multiple field transforms if you want to apply more than one field-extracting regex to the same source, source type, or host. This can be necessary in cases where the field or fields that you want to extract appear in two or more very different event patterns.

For example, the Expression column could display two values for a Uses transform extraction: access-extractions and ip-extractions. These may appear in props.conf as:

[access_combined]

REPORT-access = access-extractions, ip-extractions

In this example, access-extractions and ip-extractions are both names of field transform stanzas in transforms.conf. To work with those field transforms through Splunk Web, go to the Field transforms page.

Add new field extractions in Splunk Web

Use Splunk Web to create new field extractions.

Prerequisites

• Regular expressions and field name syntax for information about field-extracting regular expressions.
• About default fields (host, source, source type, and more) for information about hosts, sources, and sourcetypes.

Steps

1. Select Settings > Fields.
2. Click Field extractions to go to the field extractions page.
3. Click New to create a new field extraction.
4. Define a Destination app context for the field extraction. By default it will be the app context you are currently in.
5. Give the field extraction a Name, using underscores for spaces between words. In props.conf this is the <class> value for an EXTRACT or REPORT field extraction type. Note: <class> values do not have to follow field name syntax restrictions (see note below). You can use characters other than a-z, A-Z, and 0-9, and spaces are allowed. In addition <class> values are not subject to key cleaning.
6. Define the sourcetype, source, or host to which the extraction applies. Select sourcetype, source, or host and enter the value. This maps to the <spec> value in props.conf.
7. Define the extraction type.

   If you select Uses transform, enter the transform(s) involved in the Extraction/Transform field, separated by commas. The transform can then be created or updated with the Field transforms page.

   If you select Inline, enter the regular expression used to extract the field (or fields) in the Extraction/Transform field. For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command.

Example - Add a new error code field

This shows how you would define an extraction for a new err_code field. The field can be identified by the occurrence of device_id= followed by a word within brackets and a text string terminating with a colon. The field should be extracted from events related to the testlog source type.

In props.conf this extraction would look like:

[testlog]
EXTRACT-errors = device_id=\[w+\](?<err_code>[^:]+)

Here's how you would set that up through the Add new field extractions page:

Note: You can find a version of this example in Create and maintain search-time field extractions, which shows you how to set up field extractions using the props.conf file.

Create a field from a subtoken

You may run into problems if you are extracting a field value that is a subtoken--a part of a larger token. Tokens are chunks of event data that have been run through event processing prior to being indexed. During event processing, events are broken up into segments, and each segment created is a token. You will need access to the .conf files in order to create a field from a subtoken. If you create a field from a subtoken in Splunk UI, your field extraction will show up but you will be unable to use it in search. For more information, see create a field from a subtoken.

Update existing field extractions

To edit an existing field extraction, click its name in the Name column.

This takes you to a details page for that field extraction. In the Extraction/Transform field what you can do depends on the type of extraction that you are working with.

• If the field extraction is an inline extraction, you can edit the regular expression it uses to extract fields.
• If the field extraction uses one or more transforms, you can update the transform or transforms involved (put them in a comma-separated list if there is more than one.) The transforms can then be created or updated via the Field transforms page.

The field extraction above uses three transforms: wel-message, wel-eq-kv, and wel-col-kv. To find out how these transforms are set up, go to Settings > Fields > Field Transformations or use transforms.conf.

Note: Transform field extractions must include at least one valid transforms.conf field extraction stanza name.

Update field extraction permissions

When a field extraction is created through an inline method (such as IFX or a search command) it is initially only available to its creator. To make it so that other users can use the field extraction, you need to update its permissions.

Steps

1. Select Settings > Fields.
2. Click Field extractions to go to the field extractions page.
3. Find the locate field extraction on the Field extractions page and click on its Permissions

This opens the standard permission management page used in Splunk Web for knowledge objects. On this page, you can set up role-based permissions for the field extraction, and determine whether it is available to users of one specific App, or globally to users of all Apps. For more information about managing permissions with Splunk Web, see Manage knowledge object permissions.

Delete field extractions in Splunk Web

You can delete field extractions if your permissions enable you to do so. You will not be able to delete default field extractions (extractions delivered with the product and stored in the "default" directory of an app).

1. Navigate to Settings > Fields > Field extractions.
2. Click Delete for the field extraction you want to remove.

Note: Take care when deleting objects that have downstream dependencies. For example, if your field extraction is used in a search that in turn is the basis for an event type that is used by five other saved searches (two of which are the foundation of dashboard panels), all of those other knowledge objects will be negatively impacted by the removal of that extraction from the system. For more information about deleting knowledge objects, see Disable or delete knowledge objects.

Configure field extractions with .conf files

Inline and transform field extractions can be configured using .conf files. See Configure custom fields at search time.