Add a regular expression attribute
You can add a regular expression attribute to any object in your data model. Regular expression attributes turn the named groups in regular expression strings into separate data model attributes. You can arrange for the regular expression to extract attributes from the
_raw event text as well as specific field values.
1. In the Data Model Editor, open the object you'd like to add a regular expression attribute to.
- For an overview of the Data Model Editor, see Design data models and objects in this manual.
2. Click Add Attribute and select Regular Expression.
- This takes you to the Add Attributes with a Regular Expression page.
3. Under Extract From select the attribute that you want to extract the fields from.
- The Extract From list should include all of the attributes currently found in your object, with the addition of
_raw. If your regular expression is designed to extract one or more attributes from values of a specific attribute, choose that attribute from the Extract From list. On the other hand, if your regular expression is designed to parse the entire event string, choose _raw from the Extract From list.
4. Provide a Regular Expression.
- The regular expression must have at least one named group. Each named expression in the regular expression is extracted as a separate attribute. Attribute names cannot include whitespace, single quotes, double quotes, curly braces, or asterisks.
- After you provide a regular expression, the named group(s) appear under Attribute(s).
- Note: Regular expression attributes currently do not support sed mode or sed expressions.
5. (Optional) Provide different Display Name values for the attribute(s).
- Attribute Display Name values cannot include asterisk characters.
6. (Optional) Correct attribute Type values.
- They will be given String by default.
7. (Optional) Change attribute Flag values to whatever is appropriate for your needs.
8. (Optional) Click Preview to get a look at how well the attributes are represented in the object dataset.
- For more information about previewing attributes, see "Preview regular expression attribute representation," below.
9. Click Save to save your changes.
- You will be returned to the Data Model Editor. The regular expression attributes will be added to the list of calculated object attributes.
For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command. Splunk also maintains a list of useful third-party tools for writing and testing regular expressions.
Preview regular expression attribute representation
When you click Preview after defining one or more field extraction attributes, Splunk software runs the regular expression against the objects in your dataset that have the Extract From attribute you've selected (or against raw data if you're extracting from
_raw) and shows you the results. The preview results appear underneath the setup fields, in a set of four or more tabbed pages. Each of these tabs shows you information taken from a sample of events in the object dataset. You can determine how this sample is determined by selecting an option from the Sample list, such as First 1000 events or Last 24 hours. You can also determine how many events appear per page (default is 20).
If the preview doesn't return any events it could indicate that you need to adjust the regular expression, or that you have selected the wrong Extract From attribute.
The All tab
The All tab gives you a quick sense of how prevalent events that match the regular expression are in the event data. You can see an example of the All tab in action in the screen capture near the top of this topic.
It shows you an unfiltered sample of the events that have the Extract From attribute in their data. For example, if the Extract From attribute you've selected is
uri_path this tab displays only events that have a
The first column indicates whether the event matched the regular expression or not. Events that match have green checkmarks. Non-matching events have red "x" marks.
The second column displays the value of the Extract From field in the event. If the Extract From field is
_raw, the entire event string is displayed. The remaining columns display the attribute values extracted by the regular expression, if any.
The Match and Non-Match tabs
The Match and Non-Match tabs are similar to the All tab except that they are filtered to display either just events that match the regular expression or just events that do not match the regular expression. These tabs help you get a better sense of the field distribution in the sample, especially if the majority of events in the sample fall in either the matching or non-matching event set.
The attribute tab(s)
Each attribute named in the regular expression gets its own tab. An attribute tab provides a quick summary of the value distribution in the chosen sample of events. It's set up as a top values list, organized by Count and percentage. If you don't see the values you're expecting, or if the value distribution you are seeing seems off to you, this can be an indication that you need to fine-tune your regular expression.
You can also increase the sample size to find rare attribute values or values that appear further back in the past. In the example below, setting Sample to First 10,000 events uncovered a number of values for the
path attribute that do not appear when only the first 1,000 events are sampled.
The top value tables in attribute tabs are drilldown-enabled. You can click on a row to see all of the events represented by that row. For example, if you are looking at the
path attribute and you see that there are 6 events with the path
/numa/, you can click on the
/numa/ row to go to a list that displays the 6 events where
Add a lookup attribute
Add a geo IP attribute
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11