Create an unauthenticated DSP connection to Kafka
To get data from an Apache Kafka or Confluent Kafka broker into a data pipeline in Splunk Data Stream Processor (DSP), you must first create a connection. You can then use the connection in the Kafka source function to get data from Kafka into a DSP pipeline. If you have a Universal license, you can also use the connection in the Send to Kafka sink function to send data from DSP to a Kafka topic. See Licensing for the Splunk Data Stream Processor.
If you're using the data for testing purposes in a secure internal environment, you can choose to create an unauthenticated connection. Otherwise, create an SSL-authenticated connection to protect your data. See Create an SSL-authenticated DSP connection to Kafka for information about creating an SSL-authenticated connection.
Before you can create an unauthenticated Kafka connection, you must have at least one Kafka server running one of the following Kafka versions:
- Apache Kafka version 10 or higher
- Confluent Kafka version 3.0 or higher
- From the Data Stream Processor home page, click Data Management ad then select the Connections tab.
- Click Create New Connection.
- Select No-Authentication Connector for Kafka and then click Next.
- Complete the following fields:
Field Description Connection Name A unique name for your Kafka connection. Description (Optional) A description of your connection. Brokers A comma-separated list of your Kafka brokers. You must enter at least one broker. Enter each broker using the format
- Click Save.
If you're editing a connection that's being used by an active pipeline, you must reactivate that pipeline after making your changes.
You can now use your connection in a Kafka source function at the start of your data pipeline to get data from Kafka, or in a Send to Kafka sink function at the end of your pipeline to send data to Kafka.
- For instructions on how to build a data pipeline, see the Building a pipeline chapter in the Use the manual.
- For information about the source function, see Get data from Kafka in the Function Reference manual.
- For information about the sink function, see Send data to Kafka in the Function Reference manual.
Create an SSL-authenticated DSP connection to Kafka
Deserialize and preview data from Kafka in DSP
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1