Splunk® Data Stream Processor

Use the Data Stream Processor

On April 3, 2023, Splunk Data Stream Processor will reach its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.
This documentation does not apply to the most recent version of Splunk® Data Stream Processor. For documentation on the most recent version, go to the latest release.

Data Stream Processor data types

The Splunk Data Stream Processor (DSP) is strongly and implicitly typed. This means that in order to satisfy DSP's type checker, sometimes data needs to be converted or cast to different types. Use the information on this page to learn about the data types that are supported in DSP and as a reference on how to successfully convert the data types of your fields. Converting data types is useful to ensure that your arguments are being processed correctly by a function or to ensure that your data is sent in a format recognized by your data destination.

Primitive data types

The following table describes primitive data types in DSP and also includes an example of a value that you might expect see if the DSP field is typed to the specified primitive data type. In addition, when previewing data, the DSP UI shows a type icon that represents the data type of the field.

Primitive data types Description Data type UI symbol Example
Integer A 32-bit whole number. # 10
Long A 64-bit whole number. # 2147483649L
Float A 32-bit floating-point number that can also store fractional numbers. Floats can store 6 to 7 decimal digits. # 10.327000
Double A 64-bit floating-point number that can also store fractional numbers. Doubles can store up to 15 decimal digits. # 4244.546000
String A sequence or string of characters. a "Hello, World"
Bytes A byte buffer used to read and write data. B aGVsbG8gd29ybGQ=
Boolean A data type with only two possible values, either true or false. This screen image shows what the boolean data type renders as in the Data Stream Processor UI. True

Complex data types

The following table describes complex data types in DSP and also includes an example of a value that you might expect see if the DSP field is typed to the specified complex data type. In addition, when previewing data, the DSP UI shows a type icon that represents the data type of the field.

Complex data types Description Data type UI symbol Example
Collection A list of elements that share the same data type. [] A collection, or list, of integers: [1,2,3,4,5,6,7,8,9].
Map A field containing a collection of key-value pairs. {} {"name": "Henry Li", "age": 37}
Union A data type that can be any of the types in the union. Union data types include more than one type. For example, the data type of the body field in DSP is a union of all primitive and complex data types. This screen image shows what the union data type renders as in the Data Stream Processor UI. The body field in DSP events or DSP metric events which is a union type of all types.

Casting between data types

You can change between data types by using the cast and ucast functions. These functions convert the data type in the first column to the data type in the top row, with the following behavior:

From / To Integer Long Float Double String Bytes Boolean
Integer - Identity * Identity * Identity * toString * Invalid Invalid
Long Identity ** - Identity * Identity * toString * Invalid Invalid
Float Floor ** Floor ** - Identity * toString * Invalid Invalid
Double Floor ** Floor ** Identity ** - toString * Invalid Invalid
String parseInt ** parseLong ** parseFloat ** parseDouble ** - Invalid Invalid
Bytes Invalid Invalid Invalid Invalid Invalid - Invalid
Boolean Invalid Invalid Invalid Invalid toString * Invalid -
  • Data type pairs in green, which also have one asterisk, succeed for all inputs.
  • Data type pairs in yellow, which also have two asterisks, do cast, but can produce unexpected results:
    • Casting from a Long to Integer will wrap-around if the value is larger than MAX_INT or less than MIN_INT.
    • Casting from a floating-point number to an Integer or Long will result in the maximum or minimum value for the target type if the real value is too large.
    • Casting from a Double to a Float will produce +Inf or -Inf if the value is too large.
    • Casting NaN from a Double or Float will produce 0 if the output type is Integer or Long.
    • Casting a String to a numeric type will return NULL if the value is not a numeric type.
  • Data type pairs in red, which are also marked "Invalid," cannot be converted and fail the type check.

Casting unions

The runtime data type of a union can be one of many types. Casting is done to ensure that a union value is a specific type before using it as an argument in a function.

A union type can be cast to any of its contained types. If the runtime data type of the field in the union type doesn't support a specific type-cast, the value is set to NULL. For example, if a union type contains types (Long, Bytes) and is used in the LessThan expression:unionField < 50, then the following occurs:

  • When the field contains a Long, the value is used as a Long and Less Than will work as expected.
  • When the field contains Bytes, the value cannot be cast to Long, so it will be cast to NULL, and LessThan will also return NULL.
Last modified on 01 October, 2020
Send data to multiple destinations in a pipeline   Using activation checkpoints to activate your pipeline

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.1.0


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters