Splunk® Enterprise

Getting Data In

Download manual as PDF

Download topic as PDF

Process events with ingest-time eval

An ingest-time eval is a type of transform that evaluates an expression at index-time. Ingest-time eval provides much of the same functionality provided by search-time eval. The primary difference is that an ingest-time eval processes event data before indexing occurs and new fields and values that result from the evaluation are sent to indexers.

For more information on search-time eval expressions, see Use the eval command and functions in the Search Manual.

You can use ingest-time eval expressions to create new fields and perform a wide range of operations on incoming data, including mathematical, statistical, and cryptographic functions. See Evaluation functions in the Search Reference.

Why use ingest-time eval?

Ingest-time eval provides an alternative to ingest-time transformations that are difficult or impossible with regular expressions alone, such as normalizing metrics data. See Example of targeted log-to-metrics conversions in the Metrics manual.

Ingest-time eval also gives you more direct control over index-time fields. For example, you can use ingest-time eval to control exactly how an index-time field is stored in the rawdata journal of a Splunk Enterprise index. For more information, see How the indexer stores indexes in Managing Indexers and Clusters of Indexers.

Ingest-time eval syntax and usage

Ingest-time eval takes a similar format to the search-time | eval command. For more information, see eval in the Search Reference.

An ingest-time eval stanza in transforms.conf contains an INGEST_EVAL expression. For example:

[eval1]
INGEST_EVAL= field3=length (_raw) *2

You can also chain multiple comma-separated INGEST_EVAL expressions, for example:

[eval2]
INGEST_EVAL= field4=_time, field5=field4+1

For detailed usage information and examples of INGEST_EVAL, see transforms.conf.

Search-time calculated fields that use the EVAL-fieldname setting in props.conf are not available.

Data processing that occurs before indexing with ingest-time eval can impact performance.

Configure an ingest-time eval transform

You configure eval-based transforms the same way you configure other index-time transforms, using a transforms.conf file that contains the transform stanza, in conjunction with a props.conf file that references it. You must also configure a fields.conf file on the search head to enable searching of newly indexed eval fields.

To process event data with ingest-time eval, configure the following files:

Configure transforms.conf

To configure transforms.conf for ingest-time eval, follow these steps:

  1. Create a transforms.conf file in the $SPLUNK_HOME/etc/system/local directory.
  2. Add an ingest-time eval stanza that specifies the INGEST_EVAL expression. For example, the following INGEST_EVAL expression creates a new field called eval_user and populates the field with the lowercase version of the values in the username field:
    [myeval]
    INGEST_EVAL = eval_user=lower(username)
    

Configure props.conf

To configure props.conf for ingest-time eval, follow these steps:

  1. Create a props.conf in the $SPLUNK_HOME/etc/system/local directory.
  2. Add a stanza that specifies the data you want to process, such as <sourcetype>, and references the ingest-time eval stanza in transforms.conf. For example:
    [mysourcetype]
    TRANSFORMS = myeval
    

    You can mix eval-based transforms and regex-based transforms in props.conf in any order. The order in which you list the transforms determines when the transforms run relative to other stanzas in tranforms.conf. For example, TRANSFORMS = eval1,regex1,eval2,regex2 runs four different transforms.conf stanzas in that specific order.

Configure fields.conf

To configure fields.conf to enable search of ingest-time eval fields, do the following:

  1. On the search head, create a fields.conf file in the $SPLUNK_HOME/etc/system/local directory.
  2. Add a stanza that references the newly indexed field created by the INGEST_EVAL expression, as follows:
    [eval_user]
    INDEXED = True
    

For more information on how to configure index-time transforms, see Define additional indexed fields.

Examples

For basic and extended examples of eval expressions, see eval in the Search Reference.

PREVIOUS
Extract fields from files with structured data
  NEXT
About hosts

This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.3.0


Comments

Thanks, Stuartidelta01.
Appreciate your bringing this to our attention!
I’ve fixed the example and updated the page as you suggested.

Sroback splunk, Splunker
January 7, 2019

The fields.conf example references [myeval] when it should reference the name of the field not the name of the transform. In the example above, the fields.conf stanza should be [eval_user]

Stuartidelta01
November 28, 2018

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters