Example inline field extraction configurations
The following are examples of inline field extraction, using props.conf
.
Add an error code field
Create an error code field by configuring a field extraction in props.conf
. The field is identified by the occurrence of device_id=
followed by a word within brackets and a text string terminating with a colon. The field is extracted from the testlog
source type.
In props.conf
, add the following line:
[testlog] EXTRACT-errors = device_id=\[w+\](?<err_code>[^:]+)
Extract multiple fields by using one regular expression
The following is an example of a field extraction of five fields. A sample of the event data follows.
#%LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/16, changed state to down
The stanza in props.conf
for the extraction looks like this:
[syslog] EXTRACT-port_flapping = Interface\s(?<interface>(?<media>[^\d]+)(?<slot>\d+)\/(?<port>\d+))\,\schanged \sstate\sto\s(?<port_status>up|down)
Five fields are extracted as named groups: interface, media, slot, port, and port_status.
Use extracted fields to report port flapping events.
- Use tags to define event types in eventtypes.conf:
[cisco_ios_port_down] search = "changed state to down" [cisco_ios_port_up] search = "changed state to up"
- Create a report in savedsearches.conf that ties much of the above together to find port flapping and report on the results:
[port flapping] search = eventtype=cisco_ios_port_down OR eventtype=cisco_ios_port_up starthoursago=3 | stats count by interface,host,port_status | sort -count
You can then use these fields with some event types to help you find port flapping events and report on them.
Create a field from a subtoken
You may run into problems if you are extracting a field value that is a subtoken--a part of a larger token. Tokens are chunks of event data that have been run through event processing prior to being indexed. During event processing, events are broken up into segments, and each segment created is a token.
Example
Tokens are never smaller than a complete word or number. For example, you may have the word foo123
in your event. If it has been run through event processing and indexing, it is a token, and it can be a value of a field. However, if your extraction pulls out the foo
as a field value unto itself, you're extracting a subtoken. The problem is that while foo123
exists in the index, foo
does not, which means that you'll likely get few results if you search on that subtoken, even though it may appear to be extracted correctly in your search results.
Because tokens cannot be smaller than individual words within strings, a field extraction of a subtoken (a part of a word) can cause problems because subtokens will not themselves be in the index, only the larger word of which they are a part.
- (Optional) If your field value is a smaller part of a token, you must configure
props.conf
as explained here. - Add an entry to
fields.conf
.[<fieldname>] INDEXED = False INDEXED_VALUE = False
- Fill in <fieldname> with the name of your field.
- For example,
[url]
if you've configured a field named "url."
- For example,
- Set
INDEXED
andINDEXED_VALUE
to false.- This setting specifies that the value you're searching for is not a token in the index.
- Fill in <fieldname> with the name of your field.
You do not need to add this entry to fields.conf
for cases where you are extracting a field's value from the value of a default field (such as host
, source
, sourcetype
, or timestamp
) that is not indexed and therefore not tokenized.
For more information on the tokenization of event data, see About segmentation in the Getting Data In Manual.
Configure automatic key-value field extraction | Example transform field extraction configurations |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.3.0, 9.3.1, 8.1.0, 8.1.10, 8.1.11, 8.1.12
Feedback submitted, thanks!