Build field extractions with the field extractor

Use the field extractor utility to create new fields. The field extractor provides two field extraction methods: regular expression and delimiters.

The regular expression method works best with unstructured event data. You select a sample event and highlight one or more fields to extract from that event, and the field extractor generates a regular expression that matches similar events in your dataset and extracts the fields from them. The regular expression method provides several tools for testing and refining the accuracy of the regular expression. It also allows you to manually edit the regular expression.

The delimiters method is designed for structured event data: data from files with headers, where all of the fields in the events are separated by a common delimiter, such as a comma or space. You select a sample event, identify the delimiter, and then rename the fields that the field extractor finds. data that resides in a file that has headers and fields separated by specific characters

Overview of the field extractor

To help you create a new field, the field extractor takes you through a set of steps. The field extractor workflow diverges at the Select Method step, where you select the field extraction method that you want to use.

This table gives you an overview of the required steps. For detailed information about a step, click the link in the Step Title column.

Step Title	Description	Field Extraction Method
Select sample	Select the source type or source that is tied to the events that have the field (or fields) that you want to extract. Then choose a sample event that has that field (or fields).	Both
Select method	Select a field extraction method. You can have the field extractor generate a field-extracting regular expression, or you can employ delimiter-based field extraction. The choice you make depends on whether you are trying to extract fields from unstructured or structured event data.	Both
Select fields	Highlight one or more field values in the event to identify them as fields. The field extractor generates a regular expression that matches the event and extracts the field. Optionally, you can: Provide additional sample events to improve extraction accuracy. Identify required text to focus the field extraction on events that contain this text. Examine field extraction results. Update the underlying regular expression manually.	Regular expression
Rename fields	Identify the delimiter that separates all of the fields in the event, and then rename one or more of those fields.	Delimiters
Validate fields	Examine the field extraction results. Identify incorrectly extracted fields as counterexamples to improve the accuracy of the field extraction.	Regular expression
Save	Name your new field extraction, set its permissions, and save it.	Both

Access the field extractor

There are several ways to access the field extractor utility. The access method you use can determine which step of the field extractor workflow you start at.

All users can access the field extractor after running a search that returns events. You have three post-search entry points to the field extractor:

• Bottom of the fields sidebar
• All Fields dialog box
• Any event in the search results

You can also enter the field extractor:

• from the Field Extractions page in Settings.
• when you add data with a fixed source type.
• from the Splunk Web Home page (if you have Admin role privileges).

Access the field extractor from the bottom of the fields sidebar

When you use this method to access the field extractor it runs only against the set of events returned by the search that you have run. To get the full set of source types in your Splunk deployment, go to the Field Extractions page in Settings.

1. Run a search that returns events.
2. Scroll down to the bottom of the fields sidebar and click Extract New Fields.

   The field extractor starts you at the Select Sample step.

   

Access the field extractor from the All Fields dialog box

When you use this method to access the field extractor you can only extract fields from the data that has been returned by your search. To get the full set of source types in your Splunk deployment, go to the Field Extractions page in Settings.

1. Run a search that returns events.
2. At the top of the fields sidebar, click All Fields.
3. In the All Fields dialog box, click Extract new fields.

   The field extractor starts you at the at the Select Sample step.

Access the field extractor from a specific event

Use this method to select an event in your search results, and create a field extraction that:

• Extracts one or more fields found in that event.
• Is tied to the source type of that event.

When you use this method to access the field extractor, the field extractor runs against the set of events returned by the search that you have run.

1. Run a search that returns events.
2. Find an event that you want to extract fields from, and click the arrow symbol to the left of the timestamp to open it.
3. Click Event Actions, and select Extract Fields.

   The field extractor starts you at the Select Method step, in a new browser tab. You have already defined the source type and sample event.

   

Access the field extractor through the Field Extractions page in Settings

This entry method is available to all users.

1. Select Settings > Fields > Field extractions.
2. Click the Open field extractor button.

   The field extractor starts you at the Select Sample step.

Access the field extractor through the Home page

This entry method is available only to users whose roles have the edit_monitor capability, such as Admin.

On the Home page, click the extract fields link under the Add Data icon.

The field extractor starts you at the Select Sample step.

Access the field extractor after you add data

This entry method is available only to users whose roles have the edit_monitor capability, such as Admin.

After you add data to Splunk Enterprise, use the field extractor to extract fields from that data, as long as it has a fixed source type.

For example: You add a file named vendors.csv to your Splunk deployment and give it the custom source type vendors. After you save this input, you can enter the field extractor and extract fields from the events associated with the vendors source type.

Another example: You create a monitor input for the /var/log directory and select Automatic for the source type, meaning that Splunk software automatically determines the source type values of the data from that input on an event by event basis. When you save this input you do not get a prompt to extract fields from this new data input, because the events indexed from that directory can have a variety of source type values.

1. Enter the Add Data page.

   See "How do you want to add data?" in the Getting Data In manual.

2. Define a data input with a fixed source type.

   This can be an existing source type or a custom source type that you define. See "View and set source types for event data" in the Getting Data In manual.

3. Save the new data input.

   Note: Wait 30 seconds before going to the next step. This gives the Splunk software some time to index the data and get it ready for field extraction.

4. In the "File has been uploaded successfully" dialog box, click Extract Fields.

   The field extractor starts you at the Select Sample step.