Splunk® Connect for Kafka

Install and Use Splunk Connect for Kafka

Download manual as PDF

This documentation does not apply to the most recent version of KafkaConnect. Click here for the latest version.
Download topic as PDF

Security configurations for Splunk Connect for Kafka

Splunk Connect for Kafka supports the following security processes:

  • SSL
  • SASL/GSSAPI (Kerberos)
  • SASL/PLAIN
  • SASL/SCRAM-SHA-256
  • SASL/SCRAM-SHA-512

SSL

Configure workers and SinkTasks to work with your SSL secured cluster:

  1. Navigate to $KAFKA_HOME/config/connect-distributed.properties to configure the Kafka Connect worker and consumer settings to use SSL.
  2. Adjust the settings consumer.ssl.truststore.location and ssl.truststore.password to reflect your setup.
    # Worker security are located at the top level
    security.protocol=SSL
    ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks
    ssl.truststore.password=test1234
    
    # Sink security settings are prefixed with "consumer."
    consumer.security.protocol=SSL
    consumer.ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks
    consumer.ssl.truststore.password=test1234
    

    There is currently no way to change the configuration for connectors individually, but if your server supports client authentication over SSL, use a separate principal for the worker and the connectors. See Confluent's documentation on configuring workers and connectors with security for more information.

  3. Start Kafka Connect.
    ./bin/connect-distributed.sh config/connect-distributed-quickstart.properties
    

SASL/GSSAPI (Kerberos)

Configure Kafka Connect when your Kafka cluster is secured using Kerberos.

  1. Configure the Kafka Connect worker and consumer settings to use Kerberos in $KAFKA_HOME/config/connect-distributed.properties.
    # Worker security are located at the top level
    security.protocol=SASL_PLAINTEXT
    sasl.mechanism=GSSAPI
    
    # Sink security settings are prefixed with "consumer."
    consumer.sasl.mechanism=GSSAPI
    consumer.security.protocol=SASL_PLAINTEXT
    sasl.kerberos.service.name=kafka
    
  2. Modify bin/connect-distributed.sh by editing the EXTRA_ARGS environment variable.
  3. Pass in the location of the JAAS conf file. Optionally, you can specify the path to your Kerberos configuration file and set Kerberos debugging to true for troubleshooting connection issues.
    EXTRA_ARGS=${EXTRA_ARGS-'-name connectDistributed -Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/root/kafka_connect_jaas.conf -Dsun.security.krb5.debug=true'}
    

    For example, a Kafka Client JAAS file using the principal connect:

    KafkaClient {
    	com.sun.security.auth.module.Krb5LoginModule required
    	useKeyTab=true
    	storeKey=true
    	keyTab="/etc/security/keytabs/connect.keytab"
    	principal="connect/_HOST@REALM";
    };
    

    Modify the keytab and principal settings to reflect your environment.

  4. Start Kafka Connect.
    ./bin/connect-distributed.sh config/connect-distributed.properties
    

See Confluent's documentation for more information on configuring Kafka Connect using JAAS.

SASL/PLAIN

Do not run SASL/PLAIN in production without SSL.

Configure Kafka Connect worker and consumer settings to use SASL/PLAIN:

  1. Configure the Kafka Connect worker and consumer settings to use SASL/PLAIN in $KAFKA_HOME/config/connect-distributed.properties.
    # Worker security are located at the top level
    security.protocol=SASL_SSL
    sasl.mechanism=PLAIN
    
    # Sink security settings are prefixed with "consumer."
    consumer.security.protocol=SASL_SSL
    consumer.sasl.mechanism=PLAIN
    
  2. Navigate to $KAFKA_HOME/config/connect-distributed.properties and edit the EXTRA_ARGS environment variable.
  3. Pass in the location of the JAAS conf file.
    EXTRA_ARGS=${EXTRA_ARGS-'-name connectDistributed -Djava.security.auth.login.config=/root/kafka_connect_jaas.conf'}
    

    For example, a Kafka Client JAAS file for SASL/PLAIN.

    KafkaClient {
      org.apache.kafka.common.security.plain.PlainLoginModule required
      username="alice"
      password="alice-secret";
    };
    
  4. Start Kafka Connect.
    ./bin/connect-distributed.sh config/connect-distributed.properties
    

See Confluent's documentation for more information on configuring Kafka Connect using SASL/PLAIN.

SASL/SCRAM-SHA-256 and SASL/SCRAM-SHA-512

Configure the Kafka Connect worker and consumer settings to use SASL/SCRAM:

  1. Navigate to $KAFKA_HOME/config/connect-distributed.properties and make the following adjustments:
    # Worker security are located at the top level
    security.protocol=SASL_SSL
    sasl.mechanism=SCRAM-SHA-256 (or SCRAM-SHA-512)
    
    # Sink security settings are prefixed with "consumer."
    consumer.security.protocol=SASL_SSL
    consumer.sasl.mechanism=SCRAM-SHA-256 (or SCRAM-SHA-512)
    
  2. Modify bin/connect-distributed.sh by editing the EXTRA_ARGS environment variable. Pass in the location of the JAAS configuration file.
    EXTRA_ARGS=${EXTRA_ARGS-'-name connectDistributed -Djava.security.auth.login.config=/root/kafka_connect_jaas.conf'}
    

    For example, a Kafka Client JAAS file for SASL/SCRAM:

    KafkaClient {
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="alice"
      password="alice-secret";
    };
    
  3. Start Kafka Connect ./bin/connect-distributed.sh config/connect-distributed.properties

Workers and SinkTasks now work with your SASL/SCRAM secured cluster. See Confluent's documentation for more information on configuring Kafka Connect using JAAS.

PREVIOUS
Configure Splunk Connect for Kafka
  NEXT
Load balancing configurations for Splunk Connect for Kafka

This documentation applies to the following versions of Splunk® Connect for Kafka: 1.1.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters