Install Splunk Connect for Kafka

The below steps have been tested on both Apache Kafka and Confluent platform deployments.

To install Splunk Connect for Kafka, perform the following steps:

1. Navigate to the Splunk Connect for Kafka repository on github and download the latest splunk-kafka-connect-[VERSION].jar release.
2. Start your Kafka Cluster and confirm it is running.
   curl http://<KAFKA_CONNECT_HOSTNAME>:<KAFKA_CONNECT_PORT>.
   For example, curl http://localhost:8083
3. (Optional) Create a directory to store your Kafka Connect connectors. This will be used for your plugin.path setting.
4. Navigate to your /$KAFKA_HOME/config/ directory.
5. Modify the connect-distributed.properties file to include the below information:

   
   #Required configurations for Splunk Connect for Kafka
   bootstrap.servers=<BOOTSTRAP_SERVER1,BOOTSTRAP_SERVER2,BOOTSTRAP_SERVER3 >
   plugin.path=<PLUGIN_PATH>
   
   key.converter=<org.apache.kafka.connect.storage.StringConverter|org.apache.kafka.connect.json.JsonConverter|io.confluent.connect.avro.AvroConverter>
   value.converter=<org.apache.kafka.connect.storage.StringConverter|org.apache.kafka.connect.json.JsonConverter|io.confluent.connect.avro.AvroConverter>
   

6. Place the Splunk Connect for Kafka jar file in the plugin.path directory for all Kafka Connect hosts.
7. Restart your deployment's Kafka Connect services.
8. Run ./bin/connect-distributed.sh config/connect-distributed.properties to start Kafka Connect.
9. If this is a new install, create a test topic (for example, perf), and inject events into the topic using the Kafka data-gen-app or the kafka-console-producer.
10. Run the following command to confirm that Splunk Connect of Kafka has been installed and configured correctly.
    curl http://localhost:8083/connector-plugins The command will return a list of the available connectors. Splunk Connect for Kafka is available if the returned list of available connectors contains com.splunk.kafka.connect.SplunkSinkConnector.

    $KAFKA_HOME is the home directory where your Kafka Connect deployment is located. This could be the same as your Kafka home directory if you are running on a shared system.

Splunk Connect for Kafka commands

Use the following commands to check the status of Splunk Connect for Kafka, to manage connectors, and to manage tasks:

Description	Command
List active connectors	curl http://localhost:8083/connectors
Get kafka-connect-splunk connector information	curl http://localhost:8083/connectors/kafka-connect-splunk
Get kafka-connect-splunk connector configuration information	curl http://localhost:8083/connectors/kafka-connect-splunk/config
Delete kafka-connect-splunk connector	curl http://localhost:8083/connectors/kafka-connect-splunk -X DELETE
Get kafka-connect-splunk connector task information	curl http://localhost:8083/connectors/kafka-connect-splunk/tasks
Pause kafka-connect-splunk workers	curl -X PUT "http://localhost:8083/connectors/kafka-connect-splunk/pause

See the Confluent documentation for additional REST examples.