Splunk® Data Stream Processor

Function Reference

DSP 1.2.0 is impacted by the CVE-2021-44228 and CVE-2021-45046 security vulnerabilities from Apache Log4j. To fix these vulnerabilities, you must upgrade to DSP 1.2.4. See Upgrade the Splunk Data Stream Processor to 1.2.4 for upgrade instructions.

On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.

List

The following topic contains detailed descriptions of the scalar functions that you can use to modify or return lists, as well as information about how to use bracket notation to access list elements.

Accessing list elements using bracket notation

You can choose to use bracket notation instead of calling the mvindex scalar function to access elements from a list. The syntax for bracket notation is <list-name>[<element-position>], where <list-name> is the name of the list and <element-position> is a number indicating the position of the element in the list. List elements always start at position 0.

For example, assume you're streaming a record that contains the following list field named mammals:

"mammals":["elephant","tiger","lion"]

To access the tiger element in this list, you can use the bracket notation mammals[1]. The following Eval function extracts the tiger value from the list and returns it in a top-level field named species:

...| eval species = mammals[1];

You can also access elements in lists that are nested in other lists by appending the position numbers of as many nested list elements as needed, using syntax like <list-name>[parent-element-position].[nested-element-position].

As another example, consider this mammals_by_genus list field that contains nested lists:

"mammals_by_genus":[["wolf","jackal"],["tiger","lion"]]

To access the tiger element in this nested list, you can use the bracket notation mammals_by_genus[1][0], as shown in the following Eval function:

...| eval species = mammals_by_genus[1][0];

Currently, bracket notation can only be used to access elements in lists. You cannot use bracket notation to assign values to elements. For example, ... | eval mammals[1] = cheetah; results in an error. If you want to assign values to elements, use the mvappend function instead.

Accessing nested elements using bracket notation and dot notation

You can use bracket notation in combination with the dot notation for accessing map elements. See Accessing map elements using dot notation for more information about dot notation. Using dot and bracket notation, you can simplify the SPL2 expression for accessing nested lists and maps.

For example, assume you're streaming a record that contains the following list named mammals, which is nested in a zoo map, which is further nested in the z map:

"z":{"zoo":{"mammals":["elephant","tiger","lion"]}}

To extract the tiger value from the list and return it as a top-level field named species, you can write the following Eval function using dot and bracket notation:

...| eval species = z.zoo.mammals[1];

To achieve the same results using scalar functions instead of dot and bracket notation, you would need to write an Eval function with multiple expressions:

...| eval zoo=map_get(z, "zoo"), 
mammals=map_get(zoo, "mammals"),
species=mvindex(mammals, 1);

mvdedup(input)

This function takes a list input and returns a list with its duplicate values removed.

Function Input
input: collection<T>
Function Output
collection<T> with duplicates removed

1. SPL2 example

Returns ["foo","bar","biz","baz"] in new field dedup_list.

When working in the SPL View, you can write the function by using the following syntax.

... | eval dedup_list=mvdedup(["foo", "bar", "foo", "bar", "biz", "baz"]);

2. SPL2 example

Removes duplicates in a field containing a list.

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=mvdedup(mvfield);

3. SPL2 example

Alternatively, you can use named arguments.

... | eval dedup_list=mvdedup(input: ["foo", "bar", "foo", "bar", "biz", "baz"]);

iterator(input, fieldname)

For documentation on the iterator function, see Iterator.

length(input)

Returns the character length of the provided input. The input can be a map, collection, bytes, or a string.

Function Input
type<any>
Function Output
integer

SPL2 examples

Returns 4.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=length([1, 5, 3, 4]);

Alternatively, you can use named arguments.

...| eval n=length(input: [1, 5, 3, 4]);

mvappend(input)

Takes an arbitrary list of arguments, where each argument is a single string or a list of strings, and returns all elements as a single flattened list.

Function Input
input: collection<any>
Function Output
string

SPL2 examples

Returns [\"lorem\",\"lorem\",\"ipsum\",\"dolor\",\"sit\",\"amet\",\"consectetuer\"] in a new field called newlist.

When working in the SPL View, you can write the function by using the following syntax.

... | eval newlist = mvappend("lorem", "lorem", ["ipsum", "dolor"], ["sit", "amet"], "consectetuer");

Alternatively, you can use named arguments.

... | eval newlist = mvappend(input: "lorem", input: "lorem", input: ["ipsum", "dolor"], input: ["sit", "amet"], input: "consectetuer");

mvindex(input, index)

Returns the element at the list at the index.

As an alternative, you can use bracket notation to return a list element. See the Accessing list elements using bracket notation section for more information.

Function Input
input: collection<R>
This function accepts a collection of type R. R can be integers, strings, lists, etc.
index: integer
Function Output
R
This function outputs an element of the list.
Argument Input Description
input collection<R> A list of type R, where R is any type. For example, the input of this function can be a list of strings, list of numbers, list of maps, list of lists, or a list of mixed types.
index integer The index number of the element to get from the input list. Indexes start at zero. If you have 5 values in the list, the first value has an index of 0. The second values has an index of 1. Index numbers can be negative. -1 gets the last element in a list, -2 gets the second to last element in a list, and so on. If the index is out of range or does not exist, returns null.

1. SPL2 example

Returns "a".

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvindex(["a"], 0);

2. SPL2 example

Returns "a".

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvindex(["a"], -1);

3. SPL2 example

Returns "c".

When working in the SPL View, you can write the function by using the following syntax.

| eval n=mvindex(["a", "b", "c"], 2);

4. SPL2 example

Returns "a".

When working in the SPL View, you can write the function by using the following syntax.

...|eval n=mvindex(["a", "b", "c"], -3);

5. SPL2 example

Returns [3,4].

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvindex([[1,2], [3, 4]], 1);

6. SPL2 example

Returns null.

When working in the SPL View, you can write the function by using the following syntax.

| eval n=mvindex(["a"], -2);

7. SPL2 example

In this example, if the incoming record contained a field called list with [[100, 101], [0, 1, 2]] , return a new list in results with value [101, 2].

When working in the SPL View, you can write the function by using the following syntax.

...| eval results=for_each(iterator(list, "x"), mvindex(x, -1));

8. SPL2 example

Alternatively, you can use named arguments to list the arguments in any order.

...| eval n=mvindex(index: 0, input: ["a"]);

mvjoin(delimiter, values)

This function takes two arguments, a string delimiter delimiter and a list values. The function concatenates the individual values within values using the value of delimiter.

Function Input
delimiter: string
values: collection<string>
This function accepts a collection of lists, where the list is a string type.
Function Output
string

SPL2 examples

Returns foo OR bar OR baz.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvjoin(" OR ", ["foo", "bar", "baz"]);

Alternatively, you can use named arguments to list the arguments in any order.

...| eval n=mvjoin(values: ["foo", "bar", "baz"], delimiter: " OR ");

mvrange(start, end , step)

This function returns a list for a range of numbers. This function can contain up to three arguments: a starting number start, an ending number end (which is excluded from the field), and an optional step increment step, which defaults to 1. We support Splunk relative time strings as a valid step increment step. See the third SPL2 example for usage and time modifiers in the Splunk Search Reference for the full list of time modifiers.

Function Input
start: number
end: number
step: number
Function Output
collection<R>
This function outputs a collection of records of type R, where R is the same type as the function input or, when there are multiple numeric argument types, the highest of the types in the following hierarchy: Double > Float > Long > Integer.

1. SPL2 example

Returns the list [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

When working in the SPL View, you can write the function by using the following syntax.

...| eval n = mvrange(1, 11);

2. SPL2 example

Returns the list [1, 3, 5, 7, 9].

When working in the SPL View, you can write the function by using the following syntax.

...| eval n = mvrange(1, 11, 2);

3. SPL2 example

Returns the list [0L, 2000L, 4000L, 6000L, 8000L]. The elements of the returned list are type "Long" instead of "Integer", because the time modifier 2s is converted to a "Long" data type and "Long" is higher in the type priority: Double > Float > Long > Integer.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvrange(0, 10000, "2s");

4. SPL2 example

Alternatively, you can use named arguments to list the arguments in any order.

...| eval n = mvrange(start: 1, step:2, end: 11);

mvsort(input)

This function takes a list input and returns list input with the values sorted lexicographically.

Function Input
input: collection<R>
This function accepts a collection R, where R must have a lexicographic ordering.
Function Output
collection<R>
This function outputs a collection of type R, where R is the same type as the function input.

Lexicographical order sorts items based on the values used to encode the items in computer memory. In Splunk software, this is almost always UTF-8 encoding, which is a superset of ASCII.

  • Numbers are sorted before letters. Numbers are sorted based on the first digit. For example, the numbers 10, 9, 70, 100 are sorted lexicographically as 10, 100, 70, 9.
  • Uppercase letters are sorted before lowercase letters.
  • Symbols are not standard. Some symbols are sorted before numeric values. Other symbols are sorted before or after letters.
  • Null values are sorted as the string "null".

SPL2 examples

Returns the list [1, 100, 11].

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=mvsort([1, 11, 100]);

Alternatively, you can use named arguments.

...| eval n=mvsort(input: [1, 11, 100]);

split(str, delim)

This function takes two arguments, a string str and a delimiter delim to use for splitting the string. It splits the values of str on the value of delim, where delim is either a fixed string or a Java regular expression. Returns a list of strings.

Function Input
str: string to split
delim: Delimiter used to split the string. Can be either a Java regular expression or a fixed string.
Function Output
collection<string>

1. SPL2 example

This function takes the string argument "a, b, c" and splits the string on the delimiter ,. Returns ["a","b","c"].

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split("a,b,c", ",");

2. SPL2 example

This function takes the string argument "one::two::three::" and splits the string on the delimiter ::. Returns ["one","two","three",""].

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split("one::two::three::", "::");

3. SPL2 example

This function takes the string argument "a,b, c" and splits the string on the regular expression delimiter. Returns ["a","b","c"].

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split("a,b, c", /,\s*/);

4. SPL2 example

This function takes the string argument "ambM c" and splits the string on the regular expression delimiter. Returns ["a", "b", "c"].

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split("ambM c", /(?i)M\s*/);

5. SPL2 example

This function takes the string argument "a,b,c" and splits the string using an empty string delimiter. Returns ["a,b,c"] as a single entry list.

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split("a,b,c", "");

6. SPL2 example

This function takes the string argument ",," and splits the string on the delimiter ,. Returns ["","",""].

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=split(",,", ",");

7. SPL2 example

Alternatively, you can use named arguments to list the arguments in any order.

... | eval n=split(delim: "," str: "a,b,c");
Last modified on 21 April, 2021
Iterator   Map

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters