Splunk® Data Stream Processor

Function Reference

On April 3, 2023, Splunk Data Stream Processor reached its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.

All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.

Conversion

The following scalar functions convert a value of a given data type into another data type.

base64_decode(value)

Converts a Base64-encoded string to bytes. It returns null if the value is null or if the conversion fails.

Function Input
value: string
Function Output
bytes

SPL2 examples

When working in the SPL View, you can write the function by using the following syntax.

...| eval value_decoded= base64_decode(to_string(value)); 

Alternatively, you can use named arguments.

...| eval value_decoded= base64_decode(value: to_string(value)); 

base64_encode(value)

Converts a byte array value to a Base64-encoded string. It returns null if the value is null or if the conversion fails.

Function Input
value: bytes
Function Output
string

1. SPL2 example

When working in the SPL View, you can write the function by using the following syntax.

...| where body=base64_encode(to_bytes("foo"));

2. SPL2 example

Extracts the value in RecordNumber, hashes the value, and returns the value in new field HashedRecordNumber as hashed_record_number=<hashedRecordNumber>.

When working in the SPL View, you can write the function by using the following syntax.

...| eval hashedrecordnumber=concat("hashed_record_number=", base64_encode(md5(to_bytes(RecordNumber))));

3. SPL2 example

Alternatively, you can use named arguments.

...| where body=base64_encode(value: to_bytes("foo"));

cast

For documentation on the cast function, see cast.

deserialize_json_object(value)

Converts a JSON byte string into a map.

Function Input
value: bytes
Function Output
map<string, any>

SPL2 examples

Deserializes a bytes field named value.

This conversion is typically required when you're working with data from Amazon Kinesis Data Streams. Records from Amazon Kinesis Data Streams store the payload of the data in a bytes field named value, and you must deserialize this field to a more common data type such as string or map before you can use it as input in a wider variety of streaming functions.

When working in the SPL View, you can write the function by using the following syntax.

| from kinesis("kinesis-conn-id-1", "kinesis-stream-1") | eval value=deserialize_json_object(value);

Alternatively, you can use named arguments.

| from kinesis("kinesis-conn-id-1", "kinesis-stream-1") | eval value=deserialize_json_object(value: value);

from_json_array(value)

Converts a JSON string into an array of the JSON structure, including nested keys.

Function Input
value: JSON character string
Function Output
collection<any>

SPL2 examples

Returns foo.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=spath(from_json_array("[\"foo\", \"bar\", \"baz\"]"), "{0}");

Alternatively, you can use named arguments.

...| eval n=spath(from_json_array(value: "[\"foo\", \"bar\", \"baz\"]"), "{0}");

from_json_object(value)

Converts a JSON string into a map of the JSON structure, including nested keys.

Function Input
value: JSON character string
Function Output
map<string, any>

SPL2 examples

Returns {"foo":"bar"} in field jsonmap.

When working in the SPL View, you can write the function by using the following syntax.

... | eval jsonmap=from_json_object("{\"foo\": \"bar\"}");

Alternatively, you can use named arguments.

... | eval jsonmap=from_json_object(value: "{\"foo\": \"bar\"}");

gunzip(value)

Decompresses a Gzip-compressed byte array. Returns null if the byte array is null or if the decompression fails.

Function Input
value: bytes
Function Output
bytes

SPL2 examples

Decompresses the json-body field.

When working in the SPL View, you can write the function by using the following syntax.

... | eval n = gunzip(json-body);

Alternatively, you can use named arguments.

... | eval n = gunzip(value: json-body);

gzip(value)

Returns Gzip-compressed bytes. Returns null if the byte array is null or if the compression fails.

Function Input
value: bytes
Function Output
bytes (containing Gzip-compressed bytes)

SPL2 examples

Compresses the field json-body using Gzip.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n = gzip(json-body);

Alternatively, you can use named arguments.

... | eval n = gzip(value: json-body);

inet_aton(ip)

Converts a string IPv4 or IPv6 IP address and returns the address as type Long. Because IPv6 IP addresses are 128-bits, the return value is the lower 64-bits stored as type Long.

Function Input
ip: string
Function Output
long

SPL2 examples

Returns 2130706433L.

When working in the SPL View, you can write the function by using the following syntax.

...|eval ip = inet_aton("127.0.0.1");

Alternatively, you can use named arguments.

...|eval ip = inet_aton(ip: "127.0.0.1");

inet_ntoa(ip)

Converts a decimal IP address to dotted-decimal form.

Function Input
ip: long
Function Output
string

SPL2 examples

Returns 127.0.0.1.

When working in the SPL View, you can write the function by using the following syntax.

...| eval ip= inet_ntoa(2130706433L);

Alternatively, you can use named arguments.

...| eval ip= inet_ntoa(ip: 2130706433L);

parse_bool(value)

Parses a string as a boolean. Returns TRUE when the string is case-insensitive equal to "true". Returns FALSE when the string is case-insensitive equal to "false". Returns null on failure.

Function Input
value: string
Function Output
boolean

SPL2 examples

Returns true.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=parse_bool("True");

Alternatively, you can use named arguments.

...| eval n=parse_bool(value: "True");

parse_double(value)

Parses a string and returns the numeric value as a Double. Returns null if the value is null or is not a valid Double. Use this function if you want to parse numerical string values that require high amounts of precision.

Function Input
value: string
Function Output
double

SPL2 examples

Returns 1.5 as type double.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=parse_double("1.5");

Alternatively, you can use named arguments.

...| eval n=parse_double(value: "1.5");

parse_float(value)

Parses a string and returns the numeric value as a Float. Returns null if the value is null or is not a valid Float. If you want to parse numerical string values that require high amounts of precision such as timestamps, use parse_double instead.

Function Input
value: string
Function Output
float

SPL2 examples

Returns 3.1415 as a float.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=parse_float("3.1415");

Alternatively, you can use named arguments.

...| eval n=parse_float(value: "3.1415");

parse_int(value)

Parses a string as an integer. Returns null if the value is null or is not a valid integer.

Function Input
value: string
Function Output
int

SPL2 examples

Extracts HTTP-STATUS from body, parses the HTTP-STATUS string as an int, and returns the value in http_code.

When working in the SPL View, you can write the function by using the following syntax.

...| eval http_code=parse_int(map_get(extract_regex(cast(body, "string"), /HTTP-STATUS=(\d+)/), "1"));

Alternatively, you can use named arguments.

...| eval http_code=parse_int(value: map_get(extract_regex(cast(body, "string"), /HTTP-STATUS=(\d+)/), "1"));

parse_long(value)

Parses a string and returns the numeric value as Long. Returns null if the value is null or is not a valid Long.

Function Input
value: string
Function Output
long

SPL2 examples

Returns 45 as a long.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=parse_long("45");

Alternatively, you can use named arguments.

...| eval n=parse_long(value: "45");

serialize_json()

Serializes records into a JSON byte string.

Function Input
null
Function Output
bytes

SPL2 example

Serializes records into bytes, and puts the resulting JSON byte string into body.

When working in the SPL View, you can write the function by using the following syntax.

...| select serialize_json() AS body; 

serialize_json_collection(collection)

Converts a map of JSON structure into a JSON byte array.

Function Input
collection: collection<any>
Function Output
bytes

SPL2 examples

Serializes the map in attributes to bytes.

When working in the SPL View, you can write the function by using the following syntax.

... | eval attributes={"data": serialize_json_collection(["source", source, "source_type", source_type, "body", body])};

Alternatively, you can use named arguments.

... | eval attributes={"data": serialize_json_collection(collection: ["source", source, "source_type", source_type, "body", body])};

to_bytes(value)

Converts a string to a byte string. You can optionally set a character encoding.

Function Input
value: string
encoding (Optional): string
Function Output
bytes

1. SPL2 example

The following example converts the values for the foo field to bytes.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=to_bytes(foo)

2. SPL2 example

The following example converts "somestring" into bytes with UTF-8 encoding.

When working in the SPL View, you can write the function by using the following syntax.

...| eval n=to_bytes("somestring", "UTF-8");

3. SPL2 example

Alternatively, you can use named arguments to declare the arguments in any order.

...| eval n=to_bytes(encoding: "UTF-8", value: "somestring")

to_json(map)

Converts a map of a JSON object's structure to a JSON string.

Function Input
map: map<string, any>
Function Output
string

SPL2 examples

Returns {"foo":"bar"} in a new top-level field called json.

When working in the SPL View, you can write the function by using the following syntax.

... | eval json=to_json({"foo": "bar"});

Alternatively, you can use named arguments.

... | eval json=to_json(map: {"foo": "bar"});

to_regex(pattern)

Converts a DSP string type to a regex type. Use this function if you have a regular expression stored as a string and you want to pass it as an argument to a function which requires a regex type, such as match_regex. Returns null if the value is null or the conversion fails.

Function Input
pattern: string
Function Output
regex

SPL2 example

Converts ^hello from a string type to a regular expression type for use in the match_regex function.

... | eval foo=match_regex(host, to_regex("^hello"));

Alternatively, you can use a named argument to specify the pattern argument.

... | eval foo=match_regex(host, to_regex(pattern: "^hello"));

tostring(value)

Converts the type of the input field to a string type.

Function Input
value: any
The input value can be of any data type.
Function Output
string

SPL2 example

Converts body from a union type to a string type.

... | eval n = tostring(body);

Alternatively, you can use a named argument to specify the value argument.

... | eval n = tostring(value:body);

tostring(value, encoding)

Converts bytes to a string type by decoding the bytes to a string.

Function Input
value: bytes
(Optional) encoding: One of the following: "UTF-8", "UTF-16", or "UTF-32".
The character encoding method used when you encoded the field to bytes.
Function Output
string

SPL2 example

Converts a field named value from a byte type to a string type.

This conversion is typically required when you're working with data from Amazon Kinesis Data Streams. Records from Amazon Kinesis Data Streams store the payload of the data in a bytes field named value, and you must deserialize this field to a more common data type such as string or map before you can use it as input in a wider variety of streaming functions.

| from kinesis("kinesis-conn-id-1", "kinesis-stream-1") | eval value=tostring(value, "UTF-8");

Alternatively, you can use named arguments to list the arguments in any order.

| from kinesis("kinesis-conn-id-1", "kinesis-stream-1") | eval value=tostring(encoding: "UTF-8", value: value);

tostring(value, format)

Converts a numerical type to a string type.

Function Input
value: number
(Optional) format: One of the following: "hex", "commas", or "duration".
Function Output
string

The tostring function supports an optional second argument of one of the following options: "hex", "commas", or "duration".

Examples Description
tostring(X,"hex") Converts X to hexadecimal.
tostring(X,"commas") Formats X with commas. If the number includes decimals, the function rounds to nearest two decimal places.
tostring(X,"duration") Converts seconds X to the readable time format HH:MM:SS.

1. SPL2 example

Returns "1000".

When working in the SPL View, you can write the function by using the following syntax.

... | eval n=tostring(1000);

2. SPL2 example

Returns "0xF".

When working in the SPL View, you can write the function by using the following syntax.

... | eval n= tostring(15, "hex");

3. SPL2 example

Alternatively, you can use named arguments to list the arguments in any order.

... | eval n= tostring(format: "hex", value: 15);


ucast

For documentation on the ucast function, see ucast.

Last modified on 03 November, 2021
Conditional   Cryptographic

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters