Splunk® Enterprise

Knowledge Manager Manual

Download manual as PDF

Splunk Enterprise version 5.0 reached its End of Life on December 1, 2017. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Define calculated fields

The eval command enables you to devise an expression that involves automatically extracted fields and create a new field that takes the value that is the result of that expression's evaluation. (For more information and numerous eval usage examples, see the documentation for the command in the Search Reference.)

The eval command is immensely versatile and useful. But while some eval expressions are relatively simple, they often can be quite complex. If you find that you need to use a particularly long and complex eval expression on a regular basis, you may find that retyping the expression accurately in search after search is tedious business.

This is where calculated fields come to the rescue. Calculated fields enable you to define fields with eval expressions in props.conf . Then, when you're writing out a search, you can cut out the eval expression entirely and reference the field like you would any other extracted field. When you run the search, the fields will be extracted at search time and will be added to the events that include the fields in the eval expressions.

For example, take this example search from the Search Reference discussion of the eval command, which examines earthquake data and classifies quakes by their depth by creating a new Description field:

source=eqs7day-M1.csv | eval Description=case(Depth<=70, "Shallow", Depth>70 AND Depth<=300, "Mid", Depth>300 AND Depth<=700, "Deep") | table Datetime, Region, Depth, Description

Using calculated fields, you could define the eval expression for the Description field and write the search as:

source=eqs7day-M1.csv | table Datetime, Region, Depth, Description

You can now search on Description as if it is any other extracted field. Splunk software will evaluate the calculated field key for every event that contains a Depth field. You can also run searches like this:

source=eqs7day-M1.csv Description=Deep

Note: In the next section we show you how the Description calculated field would be set up in props.conf.

Create a calculated field by editing props.conf

If you have Splunk Enterprise, you can create a calculated field by adding a calculated field key to a new or preexisting props.conf stanza. You can find props.conf in $SPLUNK_HOME/etc/system/local/, or your own custom app directory in $SPLUNK_HOME/etc/apps/. (We recommend using the latter directory if you want to make it easy to transfer your data customizations to other search servers.)

Do not edit files in $SPLUNK_HOME/etc/system/default/.

For more information on configuration files, see About configuration files in the Admin Manual.

The format of a calculated field key in props.conf is as follows:

[<stanza>]
EVAL-<field_name> = <eval statement>
  • <stanza> can be:
    • <source type>, the source type of an event.
    • host::<host>, where <host> is the host for an event.
    • source::<source>, where <source> is the source for an event.
  • Calculated field keys must start with "EVAL-" (including the hyphen), but "EVAL" is not case-sensitive (can be "eVaL" for example).
  • <field_name> is case sensitive, however. This is consistent with all other Splunk field names.
  • <eval_statement> is just as flexible as it is for the eval search command. It can be evaluated to any value type, including multivals, boolean, or null.

So, to set up the Description calculated field (see the example at the top of this topic), you'd create the following stanza in props.conf:

<Stanza>
Eval-Description = case(Depth<=70, "Shallow", Depth>70 AND Depth<=300, "Mid", Depth>300 AND Depth<=700, "Deep")

Once you have properly defined a calculated field key in this manner, Splunk software calculates the field at search time for events that have the extracted fields that appear in the eval statement. Calculated field evaluation takes place after search-time field extraction and field aliasing, but before derivation of lookup fields.

Note: When you search on a calculated field, performance-wise the action is equivalent to filtering. It won't actually use the index to search, because the field values won't be found there.

Prevent overrides of existing fields

If a calculated field has the same name as a field that has been extracted by normal means, the calculated field will override the extracted field, even if the eval statement evaluates to null. You can cancel this override effect by using the coalesce function for eval in conjunction with the eval expression. coalesce takes an arbitrary number of arguments and returns the first value that is not null.

If you do not want the calculated field to override existing fields when the eval statement returns a value, use this construction:

EVAL-field = coalesce(field, <eval expression>)

If you do not want the calculated field to override existing fields when the eval statement returns null, use this construction:

EVAL-field = coalesce(<eval expression>, field)

For more information about coalesce and other eval functions see Evaluation functions in the Search Reference.

Examples demonstrating that all calculated fields are determined independently of others

When Splunk software evaluates calculated fields, it evaluates each expression as if it were independent of all of the others. This means you can't "chain" calculated field expressions, where the evaluation of one calculated field is used in the expression for another calculated field.

In the following example, for any individual event, the value of x is equivalent to the value of calculated field y because the two calculations are carried out independently of each other -- both expressions use the original value of x when they calculate x*2.

[<foo>]
EVAL-x = x * 2
EVAL-y = x * 2

If, for a specific event x=4, these calculated fields would replace the value of x with 8, and would add y=8 to the event.

Here's another slightly more "real-world" example, which involves the extracted field response_time. When it is first extracted, the value of response_time is expressed in milliseconds. We've designed two calculated fields that make use of response_time in different ways.

[<access_common>]
EVAL-response_time =  response_time/1000
EVAL-bitrate = bytes*1000/response_time

In this example two things are happening for all data with the access_common source type.

  • The first EVAL changes the value of the response_time in all sourcetype=access_common events so that it is expressed in seconds rather than milliseconds. The new "in seconds" value overrides the old "in milliseconds" value.
  • The second EVAL calculates a new field called bitrate for all sourcetype=access_common events. It is expressed in terms of bytes per second. (Note that bytes is another extracted field.)

In both calculations, response_time is initially expressed in terms of milliseconds, as both EVALs are calculated independently of the other.

Lookup fields and calculated fields

You cannot base calculated fields on lookup fields. It won't work if you try. This is because, as mentioned above, the evaluation of calculated fields takes place after search-time field extraction and field aliasing, but before derivation of lookup fields.

PREVIOUS
Configure extractions of multivalue fields with fields.conf
  NEXT
Create calculated fields with Splunk Web

This documentation applies to the following versions of Splunk® Enterprise: 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters