Splunk® Data Stream Processor

Connect to Data Sources and Destinations with DSP

Acrobat logo Download manual as PDF

Acrobat logo Download topic as PDF

Create a DSP connection to Apache Pulsar

To get data from an Apache Pulsar cluster into a data pipeline in Splunk Data Stream Processor (DSP), you must first create a connection. You can then use the connection in the Apache Pulsar source function to get data from Pulsar into a DSP pipeline.

The Apache Pulsar SSL Connector supports two-way SSL authentication where the client and server authenticate each other using the SSL/TLS protocol.


Before you can create a Pulsar connection, you must have the following:

  • An Apache Pulsar cluster version 2.5 or higher, with SSL enabled. Search for "Using TLS with KeyStore configure" in the Apache Pulsar Security manual for more information on enabling SSL in your Apache Pulsar cluster.
  • A client private key, a client certificate, and the CA certificate used to sign the client certificate.

Generate the client certificate/key pair

If you don't have a certificate and key pair for the client, signed by a CA trusted by your SSL-enabled Apache Pulsar cluster, then follow the instructions on this page to generate a new client certificate/key pair.

  1. From a command-line interface, use openssl to generate a key.
    openssl genrsa -out admin.key.pem 2048
  2. Convert the key to PKCS #8 format.
    openssl pkcs8 -topk8 -inform PEM -outform PEM -in admin.key.pem -out admin.key-pk8.pem -nocrypt
  3. Generate the certificate request. When you are asked for a common name, enter the role token that you want this key pair to use when authenticating a client.
    openssl req -config openssl.cnf -key admin.key.pem -new -sha256 -out admin.csr.pem

    The OpenSSL configuration file, openssl.cnf, contains the configuration settings and metadata that must be in the certificate request.

  4. Sign the request with the certificate authority. The client certificates use the usr_cert extension which allows the certificate to be used for client authentication.
    openssl ca -config openssl.cnf -extensions usr_cert -days 1000 -notext -md sha256 -in admin.csr.pem -out admin.cert.pem

Create the SSL-authenticated Pulsar connection in DSP

Now that you have the required certificates and keys, create a connection in the Data Stream Processor UI.

  1. From the Data Stream Processor home page, click Data Management and then select the Connections tab.
  2. Click Create New Connection.
  3. Select Apache Pulsar Connector using SSL and then click Next.
  4. Complete the following fields:
    Field Description
    Connection Name A unique name for your connection.
    Description (Optional) A description of your connection.
    Pulsar Service URL The SSL-enabled service endpoint for your Pulsar cluster. The prefix of the URL should be pulsar+ssl://
    Client Private Key The file containing the client private key, beginning with -----BEGIN RSA PRIVATE KEY----- and ending with -----END RSA PRIVATE KEY-----. This file is created as admin.key-pk8.pem in step 2 of Generate the client certificate/key pair.
    Client Certificate The file containing the client certificate, beginning with -----BEGIN CERTIFICATE----- and ending with -----END CERTIFICATE-----. This file is created as admin.cert.pem in step 4 of Generate the client certificate/key pair.
    CA Cert The file containing the original CA certificate.

    Any credentials that you upload are transmitted securely by HTTPS, encrypted, and securely stored in a secrets manager.

  5. Click Save.

    If you're editing a connection that's being used by an active pipeline, you must reactivate that pipeline after making your changes.

You can now use your connection in the Apache Pulsar source function at the start of your data pipeline to get data from Pulsar. For instructions on how to build a data pipeline, see the Building a pipeline chapter in the Use the manual. For information about the source function, see Get data from Apache Pulsar in the Function Reference manual.

Last modified on 21 June, 2021
Connecting Apache Pulsar to your DSP pipeline as a data source
Connecting Google Cloud Monitoring to your DSP pipeline

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters