Splunk® Data Stream Processor

Install and administer the Data Stream Processor

Download manual as PDF

Download topic as PDF

Configure Environment Variables

You can update or add new Data Stream Processor environment variables.

Set a default Splunk Enterprise instance

To set a default Splunk Enterprise instance for the Write to Index function, set the following environment variables.

  1. From a node in your deployment cluster, type the following.
    sudo ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_HOST https://<your-splunk-enterprise-hostname>
    sudo ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_PORT 8088   
    sudo ./set-config K8S_PIPELINES_DATA_SPLUNKD_INDEX main
    sudo ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_TOKEN <HEC token to write to the index>
    
  2. After setting the configurations, deploy your changes.
    sudo ./deploy 
    

Configure the Data Stream Processor to send data to an SSL-enabled Splunk Enterprise instance

The Splunk Data Stream Processor supports one-way SSL/TLS with the Splunk Enterprise HEC endpoints. Use these settings if your Splunk Enterprise HEC endpoints are secured via HTTPS (SSL/TLS). These settings apply globally to both Write to Index and Write to Splunk Enterprise functions. Currently, there is no support for configuring these settings outside of setting or updating environment variables before you deploy your Kubernetes pods, and you cannot change these settings on a per-connection or per-function basis.

We currently have an "all-or-nothing" approach to sending data from DSP to an SSL-enabled Splunk Enterprise instance. This means that you must have all or none of your Splunk Enterprise HEC endpoints configured to use SSL. If the K8S_PIPELINES_DATA_SPLUNKD_SSL_VALIDATION_ENABLED setting is set to true, then all DSP HEC client functions use HTTPS and require a valid certificate from the server.

  1. (Optional) Enable hostname verification. If set to true, the DSP HEC client performs a server identity check to confirm that the client is connecting to the correct server and has not been redirected by a man-in-the-middle (MITM) attack. Defaults to false.
    sudo ./set-config K8S_PIPELINES_DATA_SSL_HOSTNAME_VERIFICATION [true|false]
    
  2. (Optional) Enable SSL validation. If true, the Write to Index and Write to Splunk Enterprise functions connect to the Splunk Enterprise HEC endpoints via HTTPS and validate the server's SSL certificate. If false, the Write to Index and Write to Splunk Enterprise functions still use HTTPS but do not validate the server's SSL certificate. Defaults to true. Only set this to true if the CA certificate you used to sign your Splunk server certificates is part of the Java JRE default trust store.
    sudo ./set-config K8S_PIPELINES_DATA_SPLUNKD_SSL_VALIDATION_ENABLED [true|false]
    
  3. After setting the configurations, deploy your changes.
    sudo ./deploy
    
  4. (Optional) To update any of these settings, do the following: make the desired changes to these settings, re-deploy the cluster, and then restart all pipelines using the Write to Index and Write to Splunk Enterprise functions.

Send pipeline metrics to Splunk Enterprise for the DSP Health app

To send pipeline metrics to a Splunk Enterprise deployment via the DSP Health Application, you must set the following properties.

  1. From a node in your deployment cluster, type the following.
    sudo ./set-config K8S_FLINK_ENABLE_STATSD true
    sudo ./set-config K8S_FLINK_STATSD_HOST <your-splunk-enterprise-hostname>
    sudo ./set-config K8S_FLINK_STATSD_PORT 8125
    
  2. After setting the configurations, deploy your changes.
    sudo ./deploy
    

For additional setup instructions, see Install, configure, and use the DSP Health application.

Configure a TLS certificate for the Data Stream Processor UI

To configure TLS certificates for the DSP UI, set the following properties. The following instructions assume that you already have a TLS key and certificate.

  1. From a node in your deployment cluster, type the following.
    base64 -w0 < tls.pem > tls.pem.b64
    base64 -w0 < tls.key > tls.key.b64
    
  2. Press enter, and then type the following.
    sudo ./set-secret K8S_NGINX_CERTIFICATE_PEM_ENCODED $(< tls.pem.b64)
    sudo ./set-secret K8S_NGINX_CERTIFICATE_KEY_ENCODED $(< tls.key.b64)
    
  3. After setting the configurations, deploy your changes.
    sudo ./deploy
    
PREVIOUS
Kafka encryption and authentication using SSL
  NEXT
Network Encryption Policy

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.0.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters