Use the Field extractions page in Manager

Use the Field extractions page in Manager

Use the Field extractions page in Manager to manage search-time field extractions that have been added to props.conf. Field extractions can be added to props.conf when you use the Interactive Field Extractor, through direct props.conf edits, and when you create field extractions through the Field extractions page.

The Field extractions page enables you to:

• Review the overall set of search-time extractions that you have created or which your permissions enable you to see, for all Apps in your instance of Splunk.
• Create new search-time field extractions.
• Update permissions for field extractions. Field extractions created through the Interactive Field Extractor and the Field extractions page are initially only available to their creators until they are shared with others.
• Delete field extractions, if your app-level permissions enable you to do so, and if they are not default extractions that were delivered with the product. Default knowledge objects cannot be deleted. For more information about deleting knowledge objects, see "Curate Splunk knowledge with Manager" in this manual.

If you have "write" permissions for a particular search-time field extraction, the Field extractions page enables you to:

• Update its regular expression, if it is an inline transaction.
• Add or delete named extractions that have been defined in transforms.conf or the Field transactions page in Manager, if it uses transactions.

Note: You cannot manage index-time field extractions via Manager. We don't recommend that you change your set of index-time field extractions, but if you find that you must do so, you have to modify your props.conf and transforms.conf configuration files manually. For more information about index-time field extraction configuration, see "Configure index-time field extractions" in the Admin manual.

Navigate to the Field extractions page by selecting Manager > Fields > Field extractions.

Review search-time field extractions in Manager

To better understand how the Field extractions page in Manager displays your field extraction, it helps to understand how field extractions are set up in your props.conf and transforms.conf files.

Field extractions can be set up entirely in props.conf, in which case they are identified on the Field extractions page as inline field extractions. But some field extractions include a transforms.conf component called a field transform. To create/edit that component of the field extraction via Splunk Web, you use the Field transactions page in Manager.

For more information about transforms and the Field transforms page, see "Manage field transforms" in this manual.

For more information about field extraction setup directly in the props.conf and transforms.conf files see "Add fields at search time" in this manual.

Name column

The Name column in the Field extractions page displays the overall name of the field extraction, as it appears in props.conf. The format is:

<spec> : [EXTRACT-<name> | REPORT-<name>]

• <spec> can be:
  • <sourcetype>, the source type of an event.
  • host::<host>, where <host> is the host for an event.
  • source::<source>, where <source> is the source for an event.

EXTRACT-<name> field extractions are extractions that are wholly defined in props.conf (in other words, they do not reference a transform in transforms.conf. They are created automatically by field extractions made through IFX and certain search commands. You can also add them by making direct updates to the props.conf file. This kind of extraction is always associated with a field-extracting regular expression. On the Field extractions page, this regex appears in the Extraction/Transform column.

REPORT-<value> field extractions reference field transform stanzas in transforms.conf. This is where their field-extracting regular expressions are located. On the Field extractions page, the referenced field transform stanza is indicated in the "Extraction/Transform" stanza.

You can work with transforms in Manager through the Field transforms page. For more information see "Use the Field Transformations page in Manager" in this manual.

Type column

There are two field extraction types: inline and transforms.conf.

• Inline extractions always have EXTRACT-<name> configurations. They are identified as such because they are entirely defined within </code>props.conf</code>; they do not reference external field transforms.
• Uses transform extractions always have REPORT-<value> name configurations. As such they reference field transforms in </code>transforms.conf</code>. You can define field transforms directly in transforms.conf or via Manager using the Field transformations page.

Extraction Transform column

In the Extraction/Transform column, Manager displays different things depending on the field extraction Type.

• For inline extraction types, Manager displays the regular expression that Splunk uses to extract the field. The named group (or groups) within the regex show you what field(s) it extracts.

For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command. Splunk also maintains a list of useful third-party tools for writing and testing regular expressions.

• In the case of Uses transform extraction types, Manager displays the name of the transforms.conf field transform stanza (or stanzas) that the field extraction is linked to through props.conf. A field extraction can reference multiple field transforms if you want to apply more than one field-extracting regex to the same source, source type, or host. This can be necessary in cases where the field or fields that you want to extract appear in two or more very different event patterns.

For example, the Expression column could display two values for a Uses transform extraction: access-extractions and ip-extractions. These may appear in props.conf as:

[access_combined]

REPORT-access = access-extractions, ip-extractions

In this example, access-extractions and ip-extractions are both names of field transform stanzas in transforms.conf. To work with those field transforms through Manager, go to the Field transforms page.

Add new field extractions

Click the New button at the top of the Field extractions page to add a new field extraction. The Add New page appears.

If you know how field extractions are set up in props.conf, you should find this to be pretty simple.

All of the fields described below are required.

1. Define a Destination app context for the field extraction. By default it will be the app context you are currently in.

2. Give the field extraction a Name, using underscores for spaces between words. In props.conf this is the <name> value for an EXTRACT or REPORT field extraction type.

3. Define the sourcetype, source, or host to which the extraction applies. Select sourcetype, source, or host and enter the value. This maps to the <spec> value in props.conf.

4. Define the extraction type. If you select Uses transform enter the transform(s) involved in the Extraction/Transform field. If you select Inline enter the regular expression used to extract the field (or fields) in the Extraction/Transform field.

For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command. Splunk also maintains a list of useful third-party tools for writing and testing regular expressions.

Important: The capturing groups in your regex must identify field names that only contain alpha-numeric characters or underscores.

• Valid characters for field names are a-z, A-Z, 0-9, or _ .
• Field names cannot begin with 0-9 or _ . Leading underscores are reserved for Splunk's internal variables.
• International characters are not allowed.

Splunk applies the following "key cleaning" rules to all extracted fields, either by default or through a custom configuration:

• All characters that are not in a-z, A-Z, and 0-9 ranges are replaced with an underscore (_).
• All leading underscores and 0-9 characters are removed from extracted field names.

To disable this behavior for a specific field extraction, you have to manually modify both props.conf and transforms.conf. For more information, see "Create and maintain search-time field extractions through configuration files" in this manual.

Note: You cannot turn off key cleaning for inline field extractions (field extractions that do not require a field transform component).

Example - Add a new error code field

This shows how you would define an extraction for a new err_code field. The field can be identified by the occurrence of device_id= followed by a word within brackets and a text string terminating with a colon. The field should be extracted from events related to the testlog source type.

In props.conf this extraction would look like:

[testlog]
EXTRACT-<errors> = device_id=\[w+\](?<err_code>[^:]+)

Here's how you would set that up through the Add new field extractions page:

Note: You can find a version of this example in "Create and maintain search-time field extractions" topic in this manual, which shows you how to set up field extractions using the props.conf file.

Update existing field extractions

To edit an existing field extraction, click locate the field extraction and click its name in the Name column.

This takes you to a details page for that field extraction. In the Extraction/Transform field what you can do depends on the type of extraction that you are working with.

• If the field extraction is an inline extraction, you can edit the regular expression it uses to extract fields.
• If the field extraction uses one or more transforms, you can specify the transform or transforms involved (put them in a comma-separated list if there is more than one.) The transforms can then be created or updated via the Field transforms page.

Note: Uses transform field extractions must include at least one valid transforms.conf field extraction stanza name.

Update field extraction permissions

When a field extraction is created through an inline method (such as IFX or a search command) it is initially only available to its creator. To make it so that other users can use the field extraction, you need to update its permissions. To do this, locate the field extraction on the Field extractions page and select its Permissions link. This opens the standard permission management page used in manager for knowledge objects.

On this page you can set up role-based permissions for the field extraction, and determine whether it is available to users of one specific App, or globally to users of all Apps. For more information about managing permissions with Manager, see "Curate Splunk knowledge with Manager," in this manual.

Delete field extractions

On the Field extractions page in Manager, you can delete field extractions if your permissions enable you to do so. You won't be able to delete default field extractions (extractions that were delivered with the product and which are stored in the "default" directory of an app).

Click Delete for the field extraction that you want to remove.

Note: Take care when deleting objects that have downstream dependencies. For example, if your field extraction is used in a search that in turn is the basis for an event type that is used by five other saved searches (two of which are the foundation of dashboard panels), all of those other knowledge objects will be negatively impacted by the removal of that extraction from the system. For more information about deleting knowledge objects, see "Curate Splunk knowledge with Manager" in this manual.

• Was this topic useful?
• Post a comment