Data Stream Processor data types
The Splunk Data Stream Processor (DSP) is strongly and implicitly typed. This means that in order to satisfy DSP's type checker, sometimes data needs to be converted or cast to different types. Use the information on this page to learn about the data types that are supported in DSP and as a reference on how to successfully convert the data types of your fields. Converting data types is useful to ensure that your arguments are being processed correctly by a function or to ensure that your data is sent in a format recognized by your data destination.
Primitive data types
The following table describes primitive data types in DSP and also includes an example of a value that you might expect see if the DSP field is typed to the specified primitive data type. In addition, when previewing data, the DSP UI shows a type icon that represents the data type of the field.
Complex data types
The following table describes complex data types in DSP and also includes an example of a value that you might expect see if the DSP field is typed to the specified complex data type. In addition, when previewing data, the DSP UI shows a type icon that represents the data type of the field.
Complex data types | Description | Data type UI symbol | Example |
---|---|---|---|
Collection | A list of elements that share the same data type. | [] | A collection, or list, of integers: [1,2,3,4,5,6,7,8,9]. |
Map | A field containing a collection of key-value pairs. | {} | {"name": "Henry Li", "age": 37} |
Union | A data type that can be any of the types in the union. Union data types include more than one type. For example, the data type of the body field in DSP is a union of all primitive and complex data types. | The body field in DSP events or DSP metric events which is a union type of all types. |
Casting between data types
You can change between data types by using the cast and ucast functions. These functions convert the data type in the first column to the data type in the top row, with the following behavior:
From / To | Integer | Long | Float | Double | String | Bytes | Boolean |
---|---|---|---|---|---|---|---|
Integer | - | Identity * | Identity * | Identity * | toString * | Invalid | Invalid |
Long | Identity ** | - | Identity * | Identity * | toString * | Invalid | Invalid |
Float | Floor ** | Floor ** | - | Identity * | toString * | Invalid | Invalid |
Double | Floor ** | Floor ** | Identity ** | - | toString * | Invalid | Invalid |
String | parseInt ** | parseLong ** | parseFloat ** | parseDouble ** | - | Invalid | Invalid |
Bytes | Invalid | Invalid | Invalid | Invalid | Invalid | - | Invalid |
Boolean | Invalid | Invalid | Invalid | Invalid | toString * | Invalid | - |
- Data type pairs in green, which also have one asterisk, succeed for all inputs.
- Data type pairs in yellow, which also have two asterisks, do cast, but can produce unexpected results:
- Casting from a Long to Integer will wrap-around if the value is larger than
MAX_INT
or less thanMIN_INT
. - Casting from a floating-point number to an Integer or Long will result in the maximum or minimum value for the target type if the real value is too large.
- Casting from a Double to a Float will produce
+Inf
or-Inf
if the value is too large. - Casting NaN from a Double or Float will produce 0 if the output type is Integer or Long.
- Casting a String to a numeric type will return
NULL
if the value is not a numeric type.
- Casting from a Long to Integer will wrap-around if the value is larger than
- Data type pairs in red, which are also marked "Invalid," cannot be converted and fail the type check.
Casting unions
The runtime data type of a union can be one of many types. Casting is done to ensure that a union value is a specific type before using it as an argument in a function.
A union type can be cast to any of its contained types.
If the runtime data type of the field in the union type doesn't support a specific type-cast, the value is set to NULL
.
For example, if a union type contains types (Long, Bytes) and is used in the LessThan expression:unionField < 50
, then the following occurs:
- When the field contains a Long, the value is used as a Long and Less Than will work as expected.
- When the field contains Bytes, the value cannot be cast to Long, so it will be cast to
NULL
, and LessThan will also returnNULL
.
Send data to multiple destinations in a pipeline | Using activation checkpoints to activate your pipeline |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.1.0
Feedback submitted, thanks!