Splunk® Data Stream Processor

Function Reference

On April 3, 2023, Splunk Data Stream Processor reached its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.

All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.

Send data to Amazon Kinesis Data Streams

Use the Send to Amazon Kinesis Data Streams sink function to send data to Amazon Kinesis Data Streams.

Optionally, you can specify the Kinesis partition key for each record. If you do not provide a partition key, a hash of the payload determines the partition key.

Prerequisites

Before you can use this function, you must create a connection. See Create a connection to Amazon Kinesis Data Streams in the Connect to Data Sources and Destinations with the manual. When configuring this sink function, set the connection_id argument to the ID of that connection.

Function input schema

Accepts records with any specific schema, but the event body or payload of the incoming records must be serialized as bytes.

Required arguments

connection_id
Syntax: string
Description: The ID of the Amazon Kinesis Data Streams connection you must create before using this function.
Example in Canvas View: "conx-2b39464e-0924"
stream_name
Syntax: string
Description: The name of the Kinesis stream to send data to.
Example in Canvas View: "my-stream-123"
body
Syntax: expression<bytes>
Description: The field containing the event body or payload to send to Kinesis. You can specify the name of a field that's serialized as bytes or provide an expression that converts a field into bytes.
Example in Canvas View: to_bytes(cast(body, "string"))

Optional arguments

partition_key
Syntax: expression<string>
Description: Your Kinesis partition key. See the AWS documentation about partition keys. Defaults to null.
Example in Canvas View: "my-partition-key"

SPL2 example

When working in the SPL View, you can write the function by providing the arguments in this exact order.

...| into kinesis("879837b0-cabf-4bc2-8589-fcc4dad753e7", "my-stream-123", to_bytes(cast(body, "string")));

Alternatively, you can use named arguments to declare the arguments in any order and leave out optional arguments you don't want to declare. All unprovided arguments use their default values. The following example provides the arguments in an arbitrary order.

...| into kinesis(stream_name: "my-stream-123", connection_id: "879837b0-cabf-4bc2-8589-fcc4dad753e7", body: to_bytes(cast(body, "string")));

If you want to use a mix of unnamed and named arguments in your functions, you need to list all unnamed arguments in the correct order before providing the named arguments.

Last modified on 14 April, 2021
Send data to a Splunk index (Default for Environment)   Send data to Amazon S3

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters