Data Stream Processor data types
The Data Stream Processor supports the following data types. Use the information on this page to convert the data types of your fields. Converting data types is useful to ensure that your arguments are being processed correctly by a function or to ensure that your data is sent in a format recognized by your data sink.
When running a preview session, you can check what data type your field is by checking the type icon returned. See Monitor your pipeline with data preview and real-time metrics.
- Integer, a 32-bit whole number.
- Long, a 64-bit whole number.
- Float, a 32-bit floating-point number.
- Double, a 64-bit floating-point number.
- String, a sequence or string of characters.
- Bytes, a byte buffer used to read and write data.
- Boolean, a data type with only two possible values, either true or false.
- Collection, a group or list of variables that share the same data type.
- Map, a collection of key-value pairs.
- Union, a JSON array consisting of more than one data type.
- Regex, a regular expression that must be contained between two
/
slashes. You can add regex flags after the second/
. For example,/.*teardown.*outside.*inside/i
.
Casting between data types
To change between data types, the Data Stream Processor provides explicit cast and ucast functions. Complex data types such as maps aren't supported for casting. The following methods convert the data type in the first column to the data type in the top row:
From / To | Integer | Long | Float | Double | String | Bytes | Boolean |
---|---|---|---|---|---|---|---|
Integer | - | Identity * | Identity * | Identity * | toString * | Invalid | Invalid |
Long | Identity ** | - | Identity * | Identity * | toString * | Invalid | Invalid |
Float | Floor ** | Floor ** | - | Identity * | toString * | Invalid | Invalid |
Double | Floor ** | Floor ** | Identity ** | - | toString * | Invalid | Invalid |
String | parseInt ** | parseLong ** | parseFloat ** | parseDouble ** | - | Invalid | Invalid |
Bytes | Invalid | Invalid | Invalid | Invalid | Invalid | - | Invalid |
Boolean | Invalid | Invalid | Invalid | Invalid | toString * | Invalid | - |
- Data type pairs in green, which also have one asterisk, succeed for all inputs.
- Data type pairs in yellow, which also have two asterisks, do cast, but can produce unexpected results:
- Casting from a Long to Integer will wrap-around if the value is larger than
MAX_INT
or less thanMIN_INT
. - Casting from a floating-point number to an Integer or Long will result in the maximum or minimum value for the target type if the real value is too large.
- Casting from a Double to a Float will produce
+Inf
or-Inf
if the value is too large. - Casting NaN from a Double or Float will produce 0 if the output type is Integer or Long.
- Casting a String to a numeric type will return
NULL
if the value is not a numeric type.
- Casting from a Long to Integer will wrap-around if the value is larger than
- Data type pairs in red, which are also marked "Invalid," cannot be converted and fail the type check.
Casting unions
The runtime data type of a union can be one of many types. Casting is done to ensure that a union value is a specific type before using it as an argument in a function.
A union type can be cast to any of its contained types.
If the runtime data type of the field in the union type doesn't support a specific type-cast, the value is set to NULL
.
For example, if a union type contains types (Long, Bytes) and is used in a LessThan expression:lt(get("unionField"), 50);
, then the following occurs:
- When the field contains a Long, the value is used as a Long and LessThan works as expected.
- When the field contains Bytes, the value cannot be cast to Long, so it casts to
NULL
, and LessThan also returnsNULL
.
Create a pipeline with multiple data sources | Using activation checkpoints to activate your pipeline |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.0.0
Feedback submitted, thanks!