DSP Functions by category
Use the following section as a reference of available streaming and scalar functions.
Streaming functions
These functions are visible in the UI. Streaming functions are full nodes in the pipeline with edges that data flows through. Data streams from one node (the streaming function) through its edge to the next node (streaming function). Streaming functions can use scalar functions.
Functions | Description |
---|---|
Aggregate
|
Applies one or more aggregation functions on a stream of events in a specified time window. |
Aggregate and Trigger
|
Triggers an event output based on a custom condition over a set of aggregated events. |
Batch Events
|
Batches events before sending them to an index. |
Bin
|
Puts continuous numerical values into discrete sets, or bins, by adjusting the value of <field> so that all of the items in a particular set have the same value. |
Break Events
|
Break events using a Java regular expression as a delimiter. |
Eval
|
Calculate an expression and put the resulting value into the record as a new field. |
Extract Timestamp
|
Extract a timestamp using the first rule that matches. |
Fields
|
Select a subset of fields from a record. |
Filter
|
Keep events that pass a Boolean function. |
For Each (map)
|
Transforms each record in a stream by applying a function to each element. |
Group
|
Group a stream of records by one or more field(s) and returns a grouped stream. |
Merge Events
|
Parses data received from a universal forwarder into a stream of complete events for a Splunk Index. |
Mvexpand
|
Expand the values in a multivalue field into separate events, one event for each value in the multivalue field. |
Normalize (projection)
|
Assigns an alternative name to a field or applies scalar functions to a group of fields. |
Parse regex (rex)
|
Extract or rename fields using Java regular expression-named capturing groups. |
Rename
|
Rename one or more fields. |
Union
|
Combines streams with the same input schema into one stream with all of the events of the input streams. |
Streaming source and sink functions
These functions are visible in the UI and are source or sink functions.
Functions | Description |
---|---|
Amazon CloudWatch Metrics
|
Read data from Amazon CloudWatch Metrics. |
Amazon S3 Connector
|
Read data from Amazon S3. |
Azure Event Hubs Using SAS key
|
Read data from Azure Event Hubs Using SAS key. |
Read from Splunk Firehose
|
Reads data sent from Splunk DSP Ingest, Forwarders, and Collect API Services. |
Read from Forwarders Service
|
Reads data from the Splunk Forwarders Service. |
Read from Ingest REST API
|
Reads data from the Ingest REST API. |
Read from Apache Kafka |
Reads data from Apache Kafka without authentication. |
Read from Apache Kafka with SSL |
Reads data from Apache Kafka using SSL. |
Read from Amazon Kinesis |
Reads data from Amazon Kinesis. |
Read from Amazon Kinesis (Specify Init Stream) |
Reads data from Amazon Kinesis, specifying where to start reading from. |
Write to Index
|
Writes data to your default, pre-configured Splunk Enterprise instance. |
Write to Kafka
|
Writes data to an authenticated Kafka topic using a Kafka connection. |
Write to Kafka with SSL
|
Writes data to an authenticated Kafka topic. |
Write to Kinesis |
Send data to a Kinesis Stream using an AWS access key and secret key authentication. |
Write to Null
|
Writes data to a null sink. |
Write to Splunk Enterprise Index
|
Writes data to an external Splunk Enterprise index. |
Scalar functions
Scalar functions are not full nodes in a pipeline, but instead, help perform certain operations inside of a streaming function. You can use scalar functions to do things like addition and subtraction, comparisons, conversions, or other similar tasks.
Scalar functions operate in the context of the streaming function that they are called in.
Function Category | Function list | Description |
---|---|---|
Core functions | as
|
Returns INPUT named as NAME. |
cast
|
Converts an expression from one data type to another. | |
contains_key
|
Checks if a key exists in a map. | |
create_map
|
Creates a new map object at runtime. | |
get
|
Provides a lookup ("get") of a value in a single record. | |
if
|
Assigns an expression if the value is true, and another expression if the value is false. | |
list
|
Returns a list that contains the provided arguments. | |
literal
|
Provides a way to wrap a literal value in a callable function. | |
map_flatten
|
Accepts a nested map and flattens it with dot-concatenated field names. | |
map_get
|
Returns the value corresponding to a key in the map input. | |
map_put
|
Accepts a variable list of keys and values, which must have a nonzero and even length. | |
ucast
|
Provides a way to cast maps and collections, regardless of the data type that the map or collection may contain. | |
Aggregate functions | average
|
Returns the average of the values in the field X. |
count
|
Returns the number of non-null values in a time window. | |
max
|
Returns the maximum value in a time window. | |
mean
|
Calculates the average (mean) of values in a time window. | |
min
|
Returns the minimum value in a time window. | |
sum
|
Returns the sum of values in a time window. | |
Arithmetic functions | add
|
Adds the left and right argument and returns the result. |
divide (div)
|
Divides the left and right arguments and returns the result. | |
multiply
|
Multiplies the left and right arguments and returns the result. | |
mod
|
Returns the remainder after dividing the left and right arguments. | |
subtract
|
Subtracts the left and right arguments and returns the result. | |
Boolean scalar functions | boolean and (and)
|
Performs a Boolean AND operation on the left and right arguments and returns the result as a Boolean. |
boolean not (not)
|
Performs a Boolean NOT operation and returns the result as a Boolean. | |
boolean or (or)
|
Performs a Boolean OR operation on the left and right arguments and returns the result as a Boolean. | |
boolean xor (xor)
|
Performs a Boolean XOR operation on the left and right arguments and returns the result as a Boolean. | |
Comparison scalar functions | equals (eq)
|
Compares two values and returns LEFT == RIGHT. |
greater than (gt)
|
Compares two values and returns LEFT > RIGHT. | |
greater than or equal to (gte)
|
Compares two values and returns LEFT >= RIGHT. | |
less than (lt)
|
Compares two values and returns LEFT < RIGHT. | |
less than or equal to (lte)
|
Compares two values and returns LEFT <= RIGHT. | |
Conditional scalar functions | coalesce
|
Takes a variable number of arguments and returns the first value that is not NULL. |
in
|
Returns TRUE if one of the values in a list matches a value in the field you specify. | |
like
|
Returns TRUE if TEXT matches PATTERN. | |
null if equal (nullif)
|
Compares two fields, A and B, and returns NULL if A = B. | |
validate
|
Returns string Y corresponding to the first expression X that evaluates to FALSE. | |
Conversion scalar functions | base64_encode
|
Converts a byte array value to a Base64-encoded string. |
base64_decode
|
Converts a Base64-encoded string to bytes. | |
tostring
|
Converts a number to a string. | |
to_string
|
Converts a byte array to a String. | |
gzip
|
Returns Gzipped-compressed bytes. | |
gunzip
|
Decompresses a GZipped byte array. | |
deserialize_json_object
|
Converts a JSON byte string into a map. | |
from_json_array
|
Converts a JSON string into an array of the JSON structure, including nested keys. | |
from json object
|
Converts a JSON string into a map of the JSON structure, including nested keys. | |
inet_aton
|
Converts a string IPv4 or IPv6 IP address and returns the address as type Long. | |
inet_ntoa
|
Converts a decimal IP address to dotted-decimal form. | |
parse_bool
|
Parses a string as a boolean. | |
parse_double
|
Parses a string and returns the numeric value as a Double. | |
parse_float
|
Parses a string and returns the numeric value as a Float. | |
parse_int
|
Parses a string as an integer. | |
parse long
|
Parses a string and returns the numeric value as Long. | |
serialize_json
|
Converts the current record into a JSON byte string. | |
serialize_json
|
Converts a map of a JSON structure into a JSON byte array. | |
to_bytes
|
Converts a string to a byte string. | |
to_bytes
|
Converts a string with a character encoding you specify to a byte string. | |
to_json
|
Converts a map of a JSON object's structure to a JSON string. | |
Cryptographic scalar functions | md5
|
Computes and returns the MD5 hash of a byte value X. |
sha1
|
Computes and returns the secure hash of a byte value X based on the FIPS compliant SHA-1 hash function. | |
sha256
|
Computes and returns the secure hash of a byte value X based on the FIPS compliant SHA-256 hash function. | |
sha512
|
Computes and returns the secure hash of a byte value X based on the FIPS compliant SHA-512 hash function. | |
Date and Time scalar functions | relative_time
|
Applies a relative time specifier to a UNIX time value. |
strftime
|
This function formats a UNIX timestamp into a human-readable timestamp. | |
strptime
|
This function parses a date string into a UNIX timestamp. | |
time
|
This function returns the wall-clock time, in the UNIX time format, with millisecond resolution. | |
Mathematical scalar functions | abs
|
Takes a number and returns its absolute value. |
ceiling
|
Rounds a number up to the next highest integer. | |
exp
|
Takes a number X and returns the exponential ex. | |
floor
|
Rounds a number down to the nearest whole intenger. | |
log
|
Takes one or two numbers and returns the logarithm of the first argument X using the second argument Y as the base. | |
natural logarithm (ln)
|
Takes a number X and returns its natural logarithm. | |
pi
|
Returns the constant pi to 11 digits of precision. | |
power of base (pow)
|
Takes two numbers, X and Y, and returns XY. | |
round value (round)
|
Takes two numbers, X and Y, and returns X rounded to the amount of decimal places specified by Y. | |
round value (round)
|
Takes a number X and returns X rounded to the nearest whole number. | |
sqrt
|
Takes a number X and returns its square root. | |
Multivalue scalar functions
|
mvappend
|
Takes an arbitrary list of arguments, where each argument is a single string or a list of strings, and returns all elements as a single flattened list. |
String manipulation scalar functions | concat
|
Combines the first and second strings together. |
extract_grok
|
Extracts matching groups with a Grok-compatible pattern. | |
extract_key_value
|
Extract key-value pairs. | |
extract_regex
|
Uses a Java regular expression to extract capturing groups from the input. | |
len
|
Returns the character length of a string. | |
lower
|
Converts a string to lowercase. | |
ltrim
|
Takes two arguments X and Y and returns X with the characters in Y trimmed from the left side. | |
ltrim
|
Takes an argument X and returns X with whitespaces removed from the left side. | |
match_regex
|
Matches inputs against a pattern defined with a Java regular expression. | |
match_wildcard
|
Matches inputs against a wildcard pattern. | |
punct
|
Extracts punctuation from a string. | |
replace
|
Performs a regex replacement on a string. | |
rtrim
|
Takes two arguments X and Y and returns X with the characters in Y trimmed from the right side. | |
rtrim
|
Takes an argument X and returns X with whitespaces removed from the right side. | |
spath
|
Extract a value from a map or collection. | |
substr
|
Returns a substring of a string. Takes two arguments. | |
substr
|
Returns a substring of a string. Takes three arguments. | |
trim
|
Takes two arguments X and Y and returns X with the characters in Y trimmed from both sides. | |
trim
|
Takes an argument X and returns X with whitespaces removed from both sides. | |
upper
|
Converts a string to uppercase. | |
url_decode
|
Takes a URL string and returns the unescaped or decoded URL string. | |
url_encode
|
Encodes a string for the query string parameters in a URL. |
How to use the DSP Function Reference |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.0.0
Feedback submitted, thanks!