Splunk® Enterprise

User Manual

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Use fields to search

This topic assumes you know how to run simple searches and use the time range picker and timeline. If you're not sure, review the previous topics, beginning with Start searching.

You can learn a lot about your data from just running ad hoc searches, using nothing more than keywords and the time range. But you can't take full advantage of Splunk's more advanced searching and reporting features without understanding what fields are and how to use them. This part of the tutorial will familiarize you with:

  • default fields and other fields that Splunk automatically extracts
  • using the fields menu and fields picker to find helpful fields
  • searching with fields

Let's return to the happenings at the online Flower and Gift shop. It's your second day as a member of the Customer Support team. You spent the morning investigating some general issues and reporting the problems you found to other teams. You feel pretty good about what you've learned about the online shop and its customers, but you want to capture this and share it with your team. When you ask a coworker how you can do this, he recommends that you use fields.


What are fields?

Fields exist in machine data in many forms. Often, a field is a value (with a fixed, delimited position on the line) or a name and value pair, where there is a single value to each field name. A field can also be multivalued; that is, it appears more than once in an event and has a different value for each appearance.

In Splunk, fields are searchable name/value pairings that distinguish one event from another because not all events will have the same fields and field values. Fields enable you to write more tailored searches to retrieve the specific events that you want. Fields also enable you to take advantage of the search language, create charts, and build reports.

Some examples of fields are clientip for IP addresses accessing your Web server, _time for the timestamp of an event, and host for domain name of a server. One of the more common examples of multivalue fields is email address fields. While the "From" field will contain only a single email address, the "To" and "Cc" fields may have one or more email addresses associated with them.

For more information, read About fields in the Knowledge Manager manual.


The fields sidebar and dialog

1. Go back to the Search dashboard and search for web access activity. Select Other > Yesterday from the time range picker:

sourcetype="access_*"

2. Scroll through the search results.

If you're familiar with the access_combined format of Apache logs, you will recognize some of the information in each event, such as:

  • IP addresses for the users accessing the website.
  • URIs and URLs for the page request and referring page.
  • HTTP status codes for each page request.
  • Page request methods.


Access log fields4.3.png


As Splunk retrieves these events, the Fields sidebar updates with selected fields and interesting fields. These are the fields that Splunk extracted from your data.


Default and automatically extracted fields

Splunk extracts fields from event data twice. It extracts default and other indexed fields during event processing when that data is indexed. And it extracts a different set of fields at search time, when you run a search. Read more about "Index time versus search time" in the Admin manual.

At index time, Splunk automatically finds and extracts default fields for each event it processes. These fields include host, source, and sourcetype (which you should already be familiar with). For a complete list of the default fields, see "Use default fields" in the User manual.

Splunk also extracts certain fields at search time--when you run a search. You'll see some examples of these searches later. For more information, read the "Overview of search-time field extractions" in the Knowledge Manager manual.


Notice that default fields host, source, and sourcetype are selected fields and are displayed in your search results:


Selected fields 4.3.png


3. Scroll through Other interesting fields to see what else Splunk extracted.

You should recognize the field names that apply to the Web access logs. For example, there's clientip, method, and status. These are not default fields; they have (most likely) been extracted at search time.

4. Click the Edit link in the fields sidebar.

The Fields dialogue opens and displays all the fields that Splunk extracted.

  • Available Fields are the fields that Splunk identified from the events in your current search (some of these fields were listed under interesting fields).
  • Selected Fields are the fields you picked (from the available fields) to show in your search results (by default, host, source, and sourcetype are selected).


Fields dialogue1 4.3.png


5. Scroll through the list of Available Fields.

You're already familiar with the fields that Splunk extracted from the Web access logs based on your search. You should also see other default fields that Splunk defined--some of these fields are based on each event's timestamp (everything beginning with date_*), punctuation (punct), and location (index).

But, you should also notice other extracted fields that are related to the online store. For example, there are action, category_id, and product_id. From conversations with your coworker, you may know that these fields are:


Field name Description
action what a user does at the online shop.
category_id the type of product a user is viewing or buying.
product_id the catalog number of the product the user is viewing or buying.


6. From the Available fields list, select action, category_id, and product_id.


Fields dialogue2 4.3.png


7. Click Save.

When you return to the Search view, the fields you selected will be included in your search results if they exist in that particular event. Different events will have different fields.


Fields selected 4.3.png


The fields sidebar doesn't just show you what fields Splunk has captured from your data. It also displays how many values exist for each of these fields. For the fields you just selected, there are 2 for action, 5 for category_id, and 9 for product_id. This doesn't mean that these are all the values that exist for each of the fields--these are just the values that Splunk knows about from the results of your search.

What are some of these values?

8. Under selected fields, click action for the action field.

This opens the field summary for the action field.


Fields summary4.3.png


This window tells you that, in this set of search results, Splunk found two values for action and they are purchase and update. Also, it tells you that the action field appears in 71% of your search results. This means that three-quarters of the Web access events are related to the purchase of an item or an update (of the item quantity in the cart, perhaps).

9. Close this window and look at the other two fields you selected, category_id (what types of products the shop sells) and product_id (specific catalog names for products).

Now you know a little bit more about the information in your data relating to the online Flower and Gift shop. The online shop sells a selection of flowers, gifts, plants, candy, and balloons. Let's use these fields, category_id and product_id, to see what people are buying.

Use fields to run more targeted searches

These next two examples illustrate the difference between searching with keywords and using fields.

Example 1: Return to the search you ran to check for errors in your data. Select Other > Yesterday from the time range picker:

error OR failed OR severe OR (sourcetype=access_* (404 OR 500 OR 503))

Use fields ex1a 4.3.png


Run this search again, but this time, use fields in your search.


To search for a particular field, just type the field name and value into the search bar: fieldname=fieldvalue


The HTTP error codes are values of the status field. Now your search looks like this:

error OR failed OR severe OR (sourcetype=access_* (status=404 OR status=500 OR status=503))

Use fields ex1b 4.3.png


Notice the difference in the count of events between the two searches--because it's a more targeted search, the second search returns fewer events.

When you run simple searches based on arbitrary keywords, Splunk matches the raw text of your data. When you add fields to your search, Splunk looks for events that have those specific field/value pairs.

Also, you were actually using fields all along! Each time you searched for sourcetype=access_*, you told Splunk to only retrieve events from your web access logs and nothing else.


Example 2: Before you learned about the fields in your data, you might have run this search to see how many times flowers were purchased from the online shop:

sourcetype=access_* purchase flower*

As you typed in "flower", search assistant shows you both "flower" and "flowers' in the typeahead. Since you don't know which is the one you want, you use the wildcard to match both.


Use fields ex2a 4.3.png


If you scroll through the (many) search results, you'll see that some of the events have action=update and category_id that have a value other than flowers. These are not events that you wanted!

Run this search instead. Select Other > Yesterday from the time range picker:

sourcetype=access_* action=purchase category_id=flower*


Use fields ex2b 4.3.png

Note: Field names are case sensitive, but field values are not!

For the second search, even though you still used the wildcarded word "flower*", there is only one value of category_id that it matches (FLOWERS).

Notice the difference in the number of events that Splunk retrieved for each search; the second search returns significantly fewer events. Searches with fields are more targeted and retrieves more exact matches against your data.

As you run more searches, you want to be able to save them and reuse them or share them with your teammates. When you're ready, proceed to the next topic to learn how to save your search and share it it with others.

PREVIOUS
Change the time range
  NEXT
Save a search

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7


Comments

Stefanlasiewski, I rewrote the description. Thanks for leaving a comment!

Sophy, Splunker
April 6, 2012

> Most fields in your data exist as name and value pairs where there is one single value to each field name.<br /><br />I think this is incorrect. Most machine data out there does *not* appear as name=value pairs. Most data is a value, hopefully with a fixed, delimited position on the line. name=value pairs are extremely rare in the wild.

Stefanlasiewski
March 26, 2012

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters