Splunk® Enterprise

Knowledge Manager Manual

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

About fields

Fields are searchable name/value pairings in event data. All fields have names and can be searched with those names. ("Name/value pairings" are sometimes referred to as "key/value pairings.")

For example, look at the following search:

host=foo

In this search, host=foo is a way of indicating that you are searching for events with host fields that have values of foo. When you run this search, Splunk won't seek out events with different host field values. It also won't look for events containing other fields that share foo as a value. This means that this search gives you a more focused set of search results than you might get if you just put foo in the search bar.

As Splunk processes event data, it extracts and defines fields from that data, first at index time, and again at search time. These fields show up in the Field Picker after you run a search.

At index time Splunk extracts a small set of default fields for each event, including host, source, and sourcetype. Default fields are common to all events. Splunk can also extract custom indexed fields at index time; these are fields that you have configured for index-time extraction.

At search time Splunk automatically extracts certain fields. It:

  • automatically identifies and extracts the first 50 fields that it finds in the event data that match obvious name/value pairs, such as user_id=jdoe or client_ip=192.168.1.1, which it extracts as examples of user_id and client_ip fields. (This 50 field limit is a default that can be modified by editing the [kv] stanza in limits.conf.)
  • extracts any field explicitly mentioned in the search that it might otherwise have found though automatic extraction (but isn't among the first 50 fields identified).
  • performs custom search field extractions that you have defined, either through the Interactive Field Extractor, the Extracted fields page in Manager, configuration file edits, or search commands such as rex.

For an explanation of "search time" and "index time" see "Index time versus search time" in the Admin manual.

An example of automatic field extraction

This is an example of how Splunk automatically extracts fields without user help (as opposed to custom field extractions, which follow event-extraction rules that you define):

Say you search on sourcetype, a default field that Splunk automatically extracts for every event at index time. If your search is

sourcetype=veeblefetzer

for the past 24 hours, Splunk returns every event with a sourcetype of veeblefetzer in that time range. From this set of events, Splunk automatically extracts the first 50 fields that it can identify on its own. And it performs extractions of custom fields, based on configuration files. All of these fields will appear in the Field Picker when the search is complete.

Now, if a name/value combination like userlogin=fail appears for the first time 25,000 events into the search, and userlogin isn't among the set of custom fields that you've preconfigured, it likely won't be among the first 50 fields that Splunk finds on its own.

However, if you change your search to

sourcetype=veeblefetzer userlogin=*

Then Splunk will be smart enough to find and return all events including both the userlogin field and a sourcetype value of veeblefetzer, and it will be available in the Fields Picker along with the other fields that Splunk has extracted for this search.

Turn field discovery off to disable search time extraction and improve search performance

The Field discovery toggle enables you to disable automatic field extraction at search time, with the exception of the fields that are required to fulfill your search (so if you are searching on certain fields, it will extract those fields) and default fields such as _time, host, source, and sourcetype.

When Field discovery is set to Off, your overall search speed and performance will improve, but Splunk will not extract any fields that are not directly related to your search.

Field discovery callout.png

Field discovery is set to On by default.


Add and maintain custom search fields

To fully utilize the power of Splunk IT search, however, you need to know how to create and maintain custom search field extractions. Custom fields enable you to capture and track information that is important to your needs, but which isn't being discovered and extracted by Splunk automatically.

As a knowledge manager, you'll oversee the set of custom search field extractions created by users of your Splunk implementation, and you may define specialized groups of custom search fields yourself. This section of the Knowledge Manager manual discusses the various methods of field creation and maintenance (see the "Overview of search-time field extraction" topic) and provides examples showing how this functionality can be used.

You'll learn how to:

PREVIOUS
Understand and use the Common Information Model
  NEXT
Overview of search-time field extraction

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters