Splunk Cloud Platform

Use Edge Processors

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Edge Processor pipeline syntax

A pipeline is a Search Processing Language, version 2 (SPL2) statement that specifies what data to process, how to process it, and what destination to send the processed data to. To configure an Edge Processor to transform and route data, you must create pipelines and apply them to your Edge Processor.

The Edge Processor solution supports a subset of SPL2 commands and functions. When writing a pipeline, you can use only the commands and functions that are part of the EdgeProcessor profile. Additionally, when using a command or function that uses regular expressions, you must write the regular expressions in Regular Expression 2 (RE2) syntax. For information about the SPL2 and regular expression features that are supported for Edge Processor pipelines, see the following sections on this page:

For more information about profiles, see SPL2 compatibility profiles in the SPL2 Search Reference.

SPL2 syntax for Edge Processor pipelines

To create an Edge Processor pipeline, you must write an SPL2 statement that uses the following syntax:

$pipeline = | from $source | <processing command> | into $destination;

The following table explains how each part of this pipeline syntax determines the data processing behavior by the Edge Processor that the pipeline is applied to:

Pipeline syntax Required? Description
$pipeline Yes A variable indicating that the current SPL2 statement is a pipeline instead of a search query.
from $source Yes A command that instructs the Edge Processor to process all the incoming data that is associated with a given source type. You specify the source type using the $source setting on the Data tab of the pipeline editor.
<processing command> No One or more SPL2 commands that determine how the Edge Processor handles data. Each command must be delimited by a pipe ( | ). See the following sections on this page for more information:

If you don't include any processing commands, the Edge Processor sends data to the destination without changing the data.

into $destination Yes A command that instructs the Edge Processor to send data to a given destination. You specify the destination using the $destination setting on the Data tab of the pipeline editor.

SPL2 commands for Edge Processor pipelines

You can use the following SPL2 commands in your pipelines.

Each pipeline must include the from and into commands, as described in the SPL2 syntax for Edge Processor pipelines section on this page. You can optionally use the other commands in this list as processing commands in your pipeline.

SPL2 evaluation functions for Edge Processor pipelines

You can use the following evaluation functions in your eval and where commands:

Type of function Evaluation function
Comparison and conditional functions
Conversion functions
Date and time functions
Informational functions
JSON functions
Multivalue eval functions
Text functions

SPL2 operators for Edge Processor pipelines

You can use the following operators in your processing commands:

Type of operator Operator Documentation
Relational operators
  • equals ( = ) or ( == )
  • does not equal ( != )
  • is greater than ( > )
  • is greater than or equal to ( >= )
  • is less than ( < )
  • is less than or equal to ( <= )
See Relational operators in the SPL2 Search Manual.
Logical operators
  • AND
  • OR
  • NOT
  • XOR
See Logical operators in the SPL2 Search Manual.
Arithmetic operators
  • +
  • -
  • /
  • *
  • %
See Arithmetic operators in the SPL2 Search Reference.
Other operators IN See IN operator in the SPL2 Search Manual.

Regular expression syntax for Edge Processor pipelines

Unlike the Splunk platform, the Edge Processor solution supports Regular Expression 2 (RE2) syntax instead of Perl Compatible Regular Expressions (PCRE) syntax. When you use a command or function that uses regular expressions, such as the rex command or the replace evaluation function, you must write the regular expressions in RE2 syntax.

To learn about RE2 syntax and the differences between RE2 and PCRE, see https://github.com/google/re2/wiki/Syntax in the Google RE2 repository on GitHub.

In particular, be aware that RE2 and PCRE accept different syntax for named capture groups. Named capture groups are typically used in SPL2 statements to extract data values from the event body into fields. The RE2 syntax for a named capture group is (?P<fieldName>regular expression capture pattern), while the PCRE syntax is (?<fieldName>regular expression capture pattern).

Example: Extracting fields in an Edge Processor pipeline

To create an Edge Processor pipeline that extracts HTTP status codes from the event body into a field named httpcode, you need to include the following rex command, which contains a named capture group written in RE2 syntax:

rex field=_raw /(?P<httpcode>[1-5][0-9][0-9])/

Example: Extracting fields in a search

To define a search that extracts HTTP status codes from the event body into a field named httpcode, you need to include the following rex command, which contains a named captured group written in PCRE syntax:

rex field=_raw /(?<httpcode>[1-5][0-9][0-9])/
Last modified on 22 August, 2023
PREVIOUS
Configure global Edge Processor settings
  NEXT
Create pipelines for Edge Processors

This documentation applies to the following versions of Splunk Cloud Platform: 9.0.2209, 9.0.2303, 9.0.2305 (latest FedRAMP release)


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters