Search Tutorial

Use fields to search

You cannot take advantage of the advanced search features in Splunk Enterprise without understanding what fields are and how to use them.

About fields

When you look at the Data Summary in the search view, you see tabs for the Hosts, Sources, and Sourcetypes that described the type of data you added to your Splunk index.

These are also default fields (host, source, sourcetype) that Splunk Enterprise extracts from the data during indexing. They help to specify exactly which events you want to retrieve from the index.

What are fields?

Fields exist in machine data in many forms. Often, a field is a value (with a fixed, delimited position on the line) or a name and value pair, where there is a single value to each field name. A field can be multivalued, that is, it can appear more than once in an event and has a different value for each appearance.

Some examples of fields are clientip for IP addresses accessing your Web server, _time for the timestamp of an event, and host for domain name of a server. One of the more common examples of multivalue fields is email address fields. While the From field will contain only a single email address, the To and Cc fields have one or more email addresses associated with them.

In Splunk Enterprise, fields are searchable name and value pairings that distinguish one event from another because not all events will have the same fields and field values. Fields let you write more tailored searches to retrieve the specific events that you want.

See "About fields" in the Knowledge Manager Manual.

Extracted fields

Splunk extracts fields from event data at index-time and at search-time. See "Index time versus search time" in Managing Indexers and Clusters of Indexers.

Default and other indexed fields are extracted for each event that is processed when that data is indexed. Default fields include host, source, and sourcetype. For a list of the default fields, see "Use default fields" in the Knowledge Manager Manual.

Splunk Enterprise extracts different sets of fields, when you run a search. See "When Splunk Enterprise extracts fields" in the Knowledge Manager Manual.

You can also use the field extractor to create custom fields dynamically on your local Splunk instance. The field extractor lets you define any pattern for recognizing one or more fields in your events. See "Build field extractions with the field extractor" in the Knowledge Manager Manual.

Find and select fields

1. Go to the Search dashboard and type the following into the search bar:

sourcetype="access_*"

Search for fields that use the syntax: fieldname="fieldvalue" . Field names are case sensitive, but field values are not. You can use wildcards in field values. Quotes are required when the field values include spaces.

This search indicates that you want to retrieve only events from your web access logs and nothing else.

This search uses the wildcard access_* to match any Apache web access sourcetype, which can be access_common, access_combined, or access_combined_wcookie.

2. In the Events tab, scroll through the list of events.

If you are familiar with the access_combined format of Apache logs, you recognize some of the information in each event, such as:

• IP addresses for the users accessing the website.
• URIs and URLs for the pages requested and referring pages.
• HTTP status codes for each page request.
• GET or POST page request methods.

These are events for the Buttercup Games online store, so you might recognize other information and keywords, such as Arcade, Simulation, productId, categoryId, purchase, addtocart, and so on.

To the left of the events list is the Fields sidebar. As Splunk Enterprise retrieves the events that match your search, the Fields sidebar updates with Selected fields and Interesting fields. These are the fields that Splunk Enterprise extracted from your data.

Selected Fields are the fields that appear in your search results. The default fields host, source, and sourcetype are selected. These fields appear in all the events. The numbers next to the selected fields represent the number of different values for those fields that appear in the events returned from your search.

You can hide and show the fields sidebar by clicking Hide Fields and Show Fields.

3. Click All Fields.

In the Select Fields dialog box, you can select the fields to show in the events list.

You see more default fields, which includes fields based on each event's timestamp (everything beginning with date_*), punctuation (punct), and location (index).

Other field names apply to the web access logs. For example, clientip, method, and status. These are not default fields. They are extracted at search time.

Other extracted fields are related to the Buttercup Games online store. For example, action, categoryId, and productId.

4. Select action, categoryId, and productId and close the Select Fields dialog box.

The three fields appear under Selected Fields in the sidebar. The selected fields appear under the events in your search results if they exist in that particular event. Every event might not have the same fields.

The fields sidebar displays the number of values that exist for each field. These are the values that Splunk Enterprise indentifies from the results of your search.

5. Under Selected Fields, click the action field.

This opens the field summary for the action field.

In this set of search results, Splunk Enterprise found five values for action, and that the action field appears in 49.9% of your search results.

6. Close this window and look at the other two fields you selected, categoryId (what types of products the shop sells) and productId (specific catalog number for products).

7. Scroll through the events list.

If you click on the arrow next to an event, it opens up the list of all fields in that event.

Use this panel to view all the fields in a particular event and select or deselect individual fields for an individual event.

Run more targeted searches

The following are search examples using fields.

Example1: Search for successful purchases from the Buttercup Games store.

sourcetype=access_* status=200 action=purchase

This search uses the HTTP status field, status, to specify successful requests and the action field to search only for purchase events.

You can search for failed purchases in a similar manner using status!=200, which looks for all events where the HTTP status code is not equal to 200.

sourcetype=access_* status!=200 action=purchase

Example 2: Search for general errors.

(error OR fail* OR severe) OR (status=404 OR status=500 OR status=503)

This doesn't specify a source type. The search retrieves events in both the secure and web access logs.

Example 3: Search for how many simulation games were bought yesterday.

Select the Preset time range, Yesterday, from the time range picker and run:

sourcetype=access_* status=200 action=purchase categoryId=simulation

The count of events returned are the number of simulation games purchased.

To find the number of purchases for each type of product sold at the shop, run this search for each unique categoryId. For the number of purchases made each day of the previous week, run the search again for each time range.

Next steps

Fields also let you take advantage of the search language, create charts, and build charts. Continue to "Use the search language" to learn how to use the search language.