Define calculated fields
eval command enables you to devise an expression that involves automatically extracted fields and create a new field that takes the value that is the result of that expression's evaluation. (For more information and numerous
eval usage examples, see the documentation for the command in the Search Reference.)
eval command is immensely versatile and useful. But while some
eval expressions are relatively simple, they often can be quite complex. If you find that you need to use a particularly long and complex
eval expression on a regular basis, you may find that retyping the expression accurately in search after search is tedious business.
This is where calculated fields come to the rescue. Calculated fields enable you to define fields with eval expressions in
props.conf . Then, when you're writing out a search, you can cut out the eval expression entirely and reference the field like you would any other extracted field. When you run the search, the fields will be extracted at search time and will be added to the events that include the fields in the eval expressions.
For example, take this example search from the Search Reference discussion of the
eval command, which examines earthquake data and classifies quakes by their depth by creating a new
source=eqs7day-M1.csv | eval Description=case(Depth<=70, "Shallow", Depth>70 AND Depth<=300, "Mid", Depth>300 AND Depth<=700, "Deep") | table Datetime, Region, Depth, Description
Using calculated fields, you could define the eval expression for the
Description field and write the search as:
source=eqs7day-M1.csv | table Datetime, Region, Depth, Description
You can now search on
Description as if it is any other extracted field. Splunk software will evaluate the calculated field key for every event that contains a
Depth field. You can also run searches like this:
Note: In the next section we show you how the
Description calculated field would be set up in
Create a calculated field by editing props.conf
If you have Splunk Enterprise, you can create a calculated field by adding a calculated field key to a new or preexisting
props.conf stanza. You can find
$SPLUNK_HOME/etc/system/local/, or your own custom app directory in
$SPLUNK_HOME/etc/apps/. (We recommend using the latter directory if you want to make it easy to transfer your data customizations to other search servers.)
Do not edit files in
For more information on configuration files, see About configuration files in the Admin Manual.
The format of a calculated field key in
props.conf is as follows:
[<stanza>] EVAL-<field_name> = <eval statement>
<source type>, the source type of an event.
<host>is the host for an event.
<source>is the source for an event.
- Calculated field keys must start with "EVAL-" (including the hyphen), but "EVAL" is not case-sensitive (can be "eVaL" for example).
<field_name>is case sensitive, however. This is consistent with all other Splunk field names.
<eval_statement>is just as flexible as it is for the
evalsearch command. It can be evaluated to any value type, including multivals, boolean, or null.
So, to set up the
Description calculated field (see the example at the top of this topic), you'd create the following stanza in
<Stanza> Eval-Description = case(Depth<=70, "Shallow", Depth>70 AND Depth<=300, "Mid", Depth>300 AND Depth<=700, "Deep")
Once you have properly defined a calculated field key in this manner, Splunk software calculates the field at search time for events that have the extracted fields that appear in the eval statement. Calculated field evaluation takes place after search-time field extraction and field aliasing, but before derivation of lookup fields.
Note: When you search on a calculated field, performance-wise the action is equivalent to filtering. It won't actually use the index to search, because the field values won't be found there.
Prevent overrides of existing fields
If a calculated field has the same name as a field that has been extracted by normal means, the calculated field will override the extracted field, even if the eval statement evaluates to null. You can cancel this override effect by using the
coalesce function for eval in conjunction with the eval expression.
coalesce takes an arbitrary number of arguments and returns the first value that is not null.
If you do not want the calculated field to override existing fields when the eval statement returns a value, use this construction:
EVAL-field = coalesce(field, <eval expression>)
If you do not want the calculated field to override existing fields when the eval statement returns null, use this construction:
EVAL-field = coalesce(<eval expression>, field)
For more information about
coalesce and other
eval functions see Evaluation functions in the Search Reference.
Examples demonstrating that all calculated fields are determined independently of others
When Splunk software evaluates calculated fields, it evaluates each expression as if it were independent of all of the others. This means you can't "chain" calculated field expressions, where the evaluation of one calculated field is used in the expression for another calculated field.
In the following example, for any individual event, the value of
x is equivalent to the value of calculated field
y because the two calculations are carried out independently of each other -- both expressions use the original value of
x when they calculate
[<foo>] EVAL-x = x * 2 EVAL-y = x * 2
If, for a specific event
x=4, these calculated fields would replace the value of
8, and would add
y=8 to the event.
Here's another slightly more "real-world" example, which involves the extracted field
response_time. When it is first extracted, the value of
response_time is expressed in milliseconds. We've designed two calculated fields that make use of
response_time in different ways.
[<access_common>] EVAL-response_time = response_time/1000 EVAL-bitrate = bytes*1000/response_time
In this example two things are happening for all data with the
access_common source type.
- The first EVAL changes the value of the
sourcetype=access_commonevents so that it is expressed in seconds rather than milliseconds. The new "in seconds" value overrides the old "in milliseconds" value.
- The second EVAL calculates a new field called
sourcetype=access_commonevents. It is expressed in terms of
bytesper second. (Note that
bytesis another extracted field.)
In both calculations,
response_time is initially expressed in terms of milliseconds, as both EVALs are calculated independently of the other.
Lookup fields and calculated fields
You cannot base calculated fields on lookup fields. It won't work if you try. This is because, as mentioned above, the evaluation of calculated fields takes place after search-time field extraction and field aliasing, but before derivation of lookup fields.
Configure extractions of multivalue fields with fields.conf
Create calculated fields with Splunk Web
This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11