Splunk® Data Stream Processor

Connect to Data Sources and Destinations with DSP

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Create a connection between a Splunk forwarder and the Forwarders service

To get data from a Splunk heavy forwarder or universal forwarder into a data pipeline in the , you must first create an SSL-secured connection between the forwarder and the Forwarders service. You'll need to generate a self-signed client certificate or use an existing SSL certificate, upload it to the DSP Forwarders service, and configure your forwarder to use that certificate. You can then use the Forwarders Service source function to get data from the forwarder into a DSP pipeline.

Generate a client certificate for the Forwarders service

Generate and upload a self-signed client certificate. This certificate will be used to secure the connection between your forwarder and DSP's forwarder service.

Alternatively, you can use an existing SSL certificate. If you want to use an existing certificate, skip to Step 5 to upload the certificate to the forwarder service.

Prerequisites

  • Choose a name to identify your Splunk instance. Use this name for the generated key, certificate signing request (CSR), and PEM files.
  • Choose an email address to associate to the certificate. The email address is included in the name of the certificate.
  • If you are using an existing SSL certificate, then that SSL certificate must be in PEM format. You cannot upload more than one certificate at a time, so make sure that the file you are uploading only contains one certificate.

Steps

Follow these steps to secure the connection between the and your Splunk forwarders. If you are using an existing SSL certificate, skip to step 5.

  1. From the command line, generate a private key to sign your certificates.
    openssl genrsa -out my_forwarder.key 2048
  2. If you do not have a .rnd file in your home directory, you must use openSSL to generate a new file.
    openssl rand -writerand ~/.rnd
  3. Use your private key to generate a CSR file.
    openssl req -new -key "my_forwarder.key" -out "my_forwarder.csr" -subj "/C=US/ST=CA/O=my_organization/CN=my_forwarder/emailAddress=email@example.com"
  4. Sign your CSR file with your newly created private key and create a client certificate that expires in 2 years.
    openssl x509 -req -days 730 -in "my_forwarder.csr" -signkey "my_forwarder.key" -out "my_forwarder.pem" -sha256

    Your client certificate cannot be valid longer than 5 years.

  5. Use the Splunk Cloud Services CLI tool to format your certificate and upload your certificate to your tenant. For more information about the Splunk Cloud Services CLI, see Get started with the Splunk Cloud Services CLI.
    scloud forwarders add-certificate --input-datafile my_forwarder.pem

After the key is added to the tenant, the server response shows details about the certificate including the hash, subject, issuer, and validity dates.

By default, you can add up to 10 certificates to your tenant.

After setting up the client certificate in the Forwarders service, you'll need to configure your Splunk forwarder to use that certificate.

Configure your forwarder to use the client certificate

  1. On the host that forwards the data that you want to collect, open a shell or command prompt or PowerShell window.
  2. Concatenate your private and public keys into one file.
    cat my_forwarder.pem my_forwarder.key > my_forwarder-keys.pem
    
  3. Navigate to the configuration directory for the forwarder.
    cd $SPLUNK_HOME/etc/system/local
  4. Open outputs.conf in etc/system/local for editing and add the following stanzas. If outputs.conf does not exist, create it. See outputs.conf.spec in the Admin Manual for more information about these settings.
    [tcpout]
    defaultGroup = my_defaultGroup
    
    [tcpout:my_defaultGroup]
    server=<ip-address-of-master-node>:30001
    
    clientCert=/path/to/my_forwarder-keys.pem
    sslVerifyServerCert=false 
    
  5. (Optional) If you want to use indexer acknowledgment, set useACK to true in outputs.conf.
    useACK=true
    

If you are sending data from a large number of forwarders and you are experiencing throughput issues, you should increase the number of replicas for the ingest-s2s deployment in your Kubernetes cluster. You can increase the number of replicas by adding more nodes to your cluster. See Install the Data Stream Processor for instructions on how to join a new node to your cluster.

The forwarder starts forwarding data to DSP. You can now use the forwarder as a data source by creating a pipeline that starts with the Forwarders Service source function.

Last modified on 27 September, 2021
PREVIOUS
Connecting a Splunk forwarder to your DSP pipeline
  NEXT
Allow DSP users to use the Forwarders service

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters