About fields

About fields

Fields are searchable name/value pairings in event data. All fields have names and can be searched with those names. ("Name/value pairings" are sometimes referred to as "key/value pairings.")

For example, look at the following search:

In this search, host=foo is a way of indicating that you are searching for events with host fields that have values of foo. When you run this search, Splunk won't seek out events with different host field values. It also won't look for events containing other fields that share foo as a value. This means that this search gives you a more focused set of search results than you might get if you just put foo in the search bar.

As Splunk processes event data, it extracts and defines fields from that data, first at index time, and again at search time. These fields show up in the Field Picker after you run a search.

At index time Splunk extracts a small set of default fields for each event, including host, source, and sourcetype. Default fields are common to all events. Splunk can also extract custom indexed fields at index time; these are fields that you have configured for index-time extraction.

At search time Splunk automatically extracts certain fields. It:

• automatically identifies and extracts the first 50 fields that it finds in the event data that match obvious name/value pairs, such as user_id=jdoe or client_ip=192.168.1.1, which it extracts as examples of user_id and client_ip fields. (This 50 field limit is a default that can be modified by editing the [kv] stanza in limits.conf.)
• extracts any field explicitly mentioned in the search that it might otherwise have found though automatic extraction (but isn't among the first 50 fields identified).
• performs custom search field extractions that you have defined, either through the Interactive Field Extractor, the Extracted fields page in Manager, configuration file edits, or search commands such as rex.

For an explanation of "search time" and "index time" see "Index time versus search time" in the Admin manual.

An example of automatic field extraction

This is an example of how Splunk automatically extracts fields without user help (as opposed to custom field extractions, which follow event-extraction rules that you define):

Say you search on sourcetype, a default field that Splunk automatically extracts for every event at index time. If your search is

for the past 24 hours, Splunk returns every event with a sourcetype of veeblefetzer in that time range. From this set of events, Splunk automatically extracts the first 50 fields that it can identify on its own. And it performs extractions of custom fields, based on configuration files. All of these fields will appear in the Field Picker when the search is complete.

Now, if a name/value combination like userlogin=fail appears for the first time 25,000 events into the search, and userlogin isn't among the set of custom fields that you've preconfigured, it likely won't be among the first 50 fields that Splunk finds on its own.

However, if you change your search to

Then Splunk will be smart enough to find and return all events including both the userlogin field and a sourcetype value of veeblefetzer, and it will be available in the Fields Picker along with the other fields that Splunk has extracted for this search.

Turn field discovery off to disable search time extraction and improve search performance

The Field discovery toggle enables you to disable automatic field extraction at search time, with the exception of the fields that are required to fulfill your search (so if you are searching on certain fields, it will extract those fields) and default fields such as _time, host, source, and sourcetype.

When Field discovery is set to Off, your overall search speed and performance will improve, but Splunk will not extract any fields that are not directly related to your search.

Field discovery is set to On by default.

Add and maintain custom search fields

To fully utilize the power of Splunk IT search, however, you need to know how to create and maintain custom search field extractions. Custom fields enable you to capture and track information that is important to your needs, but which isn't being discovered and extracted by Splunk automatically.

As a knowledge manager, you'll oversee the set of custom search field extractions created by users of your Splunk implementation, and you may define specialized groups of custom search fields yourself. This section of the Knowledge Manager manual discusses the various methods of field creation and maintenance (see the "Overview of search-time field extraction" topic) and provides examples showing how this functionality can be used.

You'll learn how to:

• create and administrate search-time field extractions through Splunk Manager.
• design and manage search-time field transforms through Splunk Manager.
• use the props.conf and transforms.conf configuration files to add and maintain search-time extractions .
• configure Splunk to parse multivalue fields.

• Was this topic useful?
• Post a comment