Splunk® Enterprise

Knowledge Manager Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Field Extractor: Select Fields step

In the Select Fields step of the field extractor, highlight values in the sample event that you want the field extractor to extract as fields.

To improve the accuracy of your field extraction, you can optionally:

Identify one or more field values

Before you identify fields, make sure that they are not being extracted for this source type. See "Review existing fields".

You must identify at least one field value that you want to extract as a field.

If you identify two or more field values, the field extractor marks each value with a different highlight color.

Each time you highlight a field value and add it as a field extraction, the field extractor generates a regular expression. This regular expression matches events like the event you have selected and extracts values for the fields that you have identified.

When the field extractor generates a regular expression it runs the fields that belong to the selected source type against it and displays the results in the event listing. Events that appear to match the regular expression have a green check mark in the leftmost column while non-matching events get a red "x". Matching field values are highlit with the same colors in the sample event.

1. In the sample event, highlight a value that you want to extract as a field.

A dialog box with fields appears underneath the highlighted value.

2. Enter a name for the Field Name field.

Field names must start with a letter and contain only letters, numbers, and underscores.

3. Click Add Extraction to save the extraction.

4. (Optional) Repeat steps 1 through 3 until you identify all the values that you want to extract.

As you select more fields in an event for extraction there is a greater chance that the field extractor will be unable to generate a regular expression that can reliably extract all of the fields. You can improve the reliability of multifield extractions by adding sample events and identifying required text. You can also improve the regular expression by editing it manually.

5. (Optional) Remove or rename field extractions in the sample event by clicking on them and selecting an action of Remove or Rename. 6. Click Next to go to the Validate Fields step.

Add sample events to expand the range of the regular expression

This action is optional for the Select Fields step.

Sometimes you select a set of fields in your sample event and in the event list you find that events with those fields are not matched. This happens when the regular expression generated by the field extractor matches events with patterns similar to your sample event, but misses others that have slightly different patterns.

Try to expand the range of the regular expression by adding one of the missed events as an additional sample event. After you highlight the missed fields, the field extractor attempts to generate a new field extraction that encompasses both event patterns.

1. In the field listing table, click an event that is not matched by the regular expression but which has values for all of the fields that you are extracting from your first sample event.

Additional sample events have the greatest chance of improving the accuracy of the field extraction when their format or pattern closely matches that of the original sample event.
The sample event you select appears under the original sample event.

2. In the additional sample event, highlight the value for a field that you are extracting from the first sample event.

3. Select the correct Field Name.

You see values only for fields that you identified in the first sample event.

4. Click Add Extraction.

The field extractor attempts to expand the range of the regular expression so that it can find the field value in both event patterns. It matches the new regular expression against the event sample and displays the results in the event table.

5. (Optional) If you are extracting multiple fields, repeat steps 2 through 4 for each field.

You do not need to highlight all of the fields that are highlighted in the first sample event. For example you may find that a more reliable field extraction results when the additional sample event only highlights one of the two fields highlighted in the original sample event.

6. (Optional) Add sample events.

7. (Optional) Remove sample events by clicking the "X" next to the event.

Dsh FX select field add sample event.png

The field extractor sometimes cannot build a regular expression that matches the sample events as well as the original sample event. You can address the situation by using one of these methods.

  • Remove some of the fields you are trying to extract, if you are extracting multiple fields. This action can result in a field extraction that works across all of your selected events. The first field values you should remove are those that are embedded within longer text strings. You can set up separate field extractions for the fields that you remove.
  • Define a field extraction for each event pattern that contains the field values that you want to extract, using required text to set the extractions apart. For information about required text, see the next topic.

Identify required text to create extractions that match specific event patterns

This action is optional for the Select Fields step.

Sometimes a source type contains different kinds of events that contain the same field or fields that you want to extract. It can be difficult to design a single field extraction that matches multiple event patterns. One way to deal with this is to define a different field extraction for each event pattern.

You can focus the extraction to specific event patterns with required text. Required text behaves like a search filter. It is a string of text that must be present in the event for Splunk Enterprise to match it with the extraction.

For example, you might have event patterns for the access_combined source type that are differentiated by the strings action=addtocart, action=changequantity, action=purchase, and action=remove. You can create four extractions, one for each string, that each extract the same fields, but which have a different string for required text.

You can also use required text to make sure that a value is extracted only from specific events.

There are two limits to required text definition:

  • You can define only one string of required text for a single field extraction.
  • You cannot apply a required text string to a string of text that you highlighted as a field value, nor can you do the reverse.

1. In the sample event, highlight the text you want to require.

2. Select Require.

Dsh FX select field required text.png

3. Click Add Required Text to add the required text to the field extraction.

4. (Optional) Remove required text in the sample event by clicking it and selecting Remove Required Text.

Dsh FX select field fields defined overview.png

This example shows a field extraction that extracts fields named http_method (green) and status (yellow) and which has action=purchase defined as required text. In the field listing table, the first two events do not match the extraction, because they do not have the required text. The third event matches the regular expression and has the required text. It has highlighting that shows the extracted fields.

Preview the results of the field extraction

This action is optional for the Select Fields and Validate Fields steps.

The event list has features that you can use to inspect the accuracy of the field extraction. The list displays all of the events in the sample for the source type, by default.

  • Use the left-most column to identify which events match the regular expression and which events do not.
  • If the regular expression matches a small percentage of the sample events, toggle the view to Matches to review the nonmatching events from the list. You can also select Non-Matches to see only the events that fail to match the regular expression.
  • Click a field tab to see statistics for that field. Each field tab displays a bar chart showing the count of each value found for the field in the event sample, organized from highest to lowest.

Dsh FX select field preview of status field.png

  • Click a value in the chart to filter the field listing table on that value. For example, in the status chart, a click on the 503 value causes the field extractor to return to the main Preview field list view, with the filter set to status=503. It lists only events with that status value.

Note If you find field values that have been incorrectly extracted, you can submit them as counterexamples in See Validate Fields, the next field extractor step.

Manually edit the regular expression

This action is optional for the Select Fields and Validate Fields steps.

You can manually edit the regular expression. However, doing this takes you out of the field extractor workflow. When you save your changes you go to the final Save step of the field extractor, where you name the field extraction, set its permissions, and save it. You cannot validate or fine-tune the extraction before you save it.

1. Click Show Regular Expression.

2. Click Edit the Regular Expression.

If you do not want to exit the field extractor workflow, click the Back button at the top left of the page to return to the workflow. This button disappears after you begin editing the regular expression.

3. Edit the regular expression.

4. Click Preview to match your edited extraction against the sample events.

Repeat steps 3 and 4 until the regular expression is matching events and extracting fields appropriately.

5. Click Save to save your new field extraction.

When you enter the Save step, click Back to continue editing the regular expression. The Back button disappears after you enter a name for the extraction or make permissions choices.

Dsh FX select field manual regex.png

See "About Splunk Enterprise regular expressions," in this manual.

Review existing fields

This is an optional action for the Select Fields and Validate Fields steps.

The field you want to extract may already be extracted for the source type that you have selected. You can determine whether this is so, and if so, whether it is being extracted from the event pattern that you want to extract the field from.

1. Click Existing fields in the upper right of the screen.

The Existing fields button appears in the Select Fields, Validate Fields, and Save steps of the field extractor.
The Fields sidebar opens. If any fields are extracted for the source type, they appear in a table. If you do not see the field that you have extracted, click the X in the corner to close the sidebar.

Dsh FX existing fields.png

The field name may appear multiple times with different Pattern Name values.

2. If the field that you want to extract appears in the table, click open to view detail information about its field extraction.

This opens a page in a new tab. In this page you can inspect the regular expression for the field extraction, the events that it matches, and the field values that it extracts.

3. Review the Regular Expression, the events it matches, and the field values it extracts.

If the events match the type of event event pattern that you are hoping to extract the field from and you can see that the field values are extracted correctly, you do not need to create a new field extraction.
If the field extraction matches a different event pattern than the one you want to extract the field from, you can create a new extraction for the field as long as it has a unique Pattern Name.
PREVIOUS
Field Extractor: Select Sample Event step
  NEXT
Field Extractor: Validate Fields step

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters