Use fields to search
To take advantage of the advanced search features in the Splunk software, you must understand what fields are and how to use them.
What are fields?
Fields exist in machine data in many forms. Often, a field is a value with a fixed, delimited position on a line, or a name and value pair, where there is a single value to each field name. A field can be multivalued, that is, it can appear more than once in an event and have a different value for each appearance.
- Some examples of fields are
clientipfor IP addresses accessing your Web server,
_timefor the timestamp of an event, and
hostfor domain name of a server.
- One of the more common examples of multivalue fields is email address fields. While the
Fromfield will contain only a single email address, the
Ccfields have one or more email addresses associated with them.
Fields are searchable name and value pairings that distinguish one event from another. Not all events have the same fields and field values. Use fields to write more tailored searches to retrieve the specific events that you want.
The Splunk software extracts fields from event data at index time and at search time.
- Index time
- The time span from when the Splunk software receives new data to when the data is written to an index. During index time, the data is parsed into segments and events. Default fields and timestamps are extracted, and transforms are applied.
- Search time
- The period of time beginning when a search is launched and ending when the search finishes. During search time, certain types of event processing take place, such as search time field extraction, field aliasing, source type renaming, event type matching, and so on.
The default fields and other indexed fields are extracted for each event when your data is indexed.
Search with fields
When you search for fields, you use the syntax
- Field names are case sensitive, but field values are not.
- You can use wildcards in field values.
- Quotation marks are required when the field values include spaces.
- Click Search in the App bar to start a new search.
- To search the sourcetype field for any values that begin with access_, run the following field-value search.
- This search indicates that you want to retrieve only events from your web access logs and nothing else.
- This search uses a wildcard character,
access_*, in the field value to match any Apache web access
sourcetype. The source types can be access_common, access_combined, or access_combined_wcookie.
- Scroll through the list of events in your search results.
- If you are familiar with the access_combined format of Apache logs, you might recognize some of the information in each event, such as:
- IP addresses for the users accessing the website.
- URIs and URLs for the pages requested and referring pages.
- HTTP status codes for each page request.
- GET or POST page request methods.
- These are events for the Buttercup Games online store, so you might recognize other information and keywords in the search results, such as Arcade, Simulation, productId, categoryId, purchase, addtocart, and so on.
- To the left of the events list is the Fields sidebar. As events are retrieved that match your search, the Fields sidebar updates with Selected fields and Interesting fields. These are the fields that the Splunk software extracts from your data.
- When you first run a search the Selected Fields list contains the default fields host, source, and sourcetype. The default fields appear in every event.
- Interesting Fields are fields that appear in at least 20% of the events.
Specify additional selected fields
You can designate other fields to appear in the Selected Fields list. When you add a field to the Selected Fields list, the field name and field value are included in the search results.
- To add fields to the Selected Fields list, click All Fields.
- The Select Fields dialog box shows a list of fields in your events. The # of Values column shows the number of unique values for each field in the events. Because your search criteria specifies the source type, the sourcetype field has just 1 value.
- The list contains additional default fields, fields that are unique to the source type, and fields that are related to the Buttercup Games online store.
- In addition to the three default fields that appear automatically in the list of Selected Fields, there are other default fields that are created when your data is indexed. For example, fields that are based on the event
date_*). The field that identifies data that contains punctuation is the
punctfield. The field that specifies the location of the data in your Splunk deployment is the
- Other field names apply to the web access logs that you are searching. For example, the
statusfields. These are not default fields. They are extracted at search time.
- Other extracted fields are related to the Buttercup Games online store. For example,
- The three fields that you selected appear under Selected Fields in the Fields sidebar. The selected fields also appear in the events in your search results, if those fields exist in that particular event. Every event might not have the same fields.
Identifying field values
The Fields sidebar displays the number of unique values for each field in the events. These are the same numbers that appear in the Select Fields dialog box.
- Under Selected Fields, notice the number 5 next to the
- Click the
- In this set of search results there are five values for
actionfield appears in 49.9% of your search results.
- Close the action field summary window.
- Review the other two fields you added to the Selected fields. The
categoryIdfield identifies the types of games or other products that are sold by the Buttercup Games online store. The
productIdfield contains the catalog numbers for each product.
- Scroll through the events list.
- The i column contains event information. In the i column, click the arrow ( > ) next to an event to expand the event information.
- You can use this expanded panel to view all the fields in a particular event, and select or deselect individual fields for an individual event.
Run targeted searches
The following examples are searches that use fields.
Search for successful purchases
Search for successful purchases from the Buttercup Games store.
- Start a new search.
- Run the following search.
sourcetype=access_* status=200 action=purchase
- This search uses the HTTP status field,
status, to specify successful requests and the
actionfield to search only for purchase events.
- You can search for failed purchases in a similar manner using
status!=200, which looks for all events where the HTTP status code is not equal to 200.
- Change the
statusportion of the search and run the search again.
sourcetype=access_* status!=200 action=purchase
Search for errors
The way that errors are designed in events varies from source to source. To search for errors, your search must specify these different designations.
Use Boolean operators to specify different error criteria. Use parenthesis to group parts of your search string.
- Start a new search.
- Run the following search.
(error OR fail* OR severe) OR (status=404 OR status=500 OR status=503)
- This search does not specify a source type. The search retrieves events from both the secure log files and the web access log files.
Search for sales of a specific product
Search for how many simulation style games were bought yesterday.
- In the time range picker, select Yesterday from the Presets list.
- If you downloaded the
tutorialdata.zipfile more than one day ago, there are no events that have a timestamp for yesterday. Instead, change the time range picker to All time and run the previous search. In the search results, look at the dates. Use the Date Range option in the time range picker to specify one of the dates in your results.
- Run the following search.
sourcetype=access_* status=200 action=purchase categoryId=simulation
- As you type the search, the Search Assistant shows you a list of your previous searches that start with "sourcetype". You can select the search that you ran earlier to search for successful purchases. Then add
categoryId=simulationto the end of that search.
- The count of events returned are the number of simulation games purchased.
- Find the number of purchases for each type of product sold at the shop.
- Locate the unique categoryId values by clicking on the categoryId field in the Selected Fields list.
- Run the search in step 2 for each unique categoryId value.
- For the number of purchases made each day of the previous week, run the search again for each time range.
You can use your knowledge about fields to take advantage of the Splunk search processing language to generate statistics and build charts.
Let's learn how to use the search language.
In the Knowledge Manager Manual
Basic searches and search results
Use the search language
This documentation applies to the following versions of Splunk® Enterprise: 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10