Configure Environment Variables

You can update or add new Data Stream Processor environment variables.

Prerequisites

• You must have system administrator (root) permissions to run DSP installation or cluster management commands. If you do not have these permissions, you can use the sudo command.

Set a default Splunk Enterprise instance

To set a default Splunk Enterprise instance for the Write to Index function, set the following environment variables.

1. From a node in your deployment cluster, type the following.

   ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_HOST https://<your-splunk-enterprise-hostname>
   ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_PORT 8088   
   ./set-config K8S_PIPELINES_DATA_SPLUNKD_INDEX main
   ./set-config K8S_PIPELINES_DATA_SPLUNKD_HEC_TOKEN <HEC token to write to the index>
   

2. After setting the configurations, deploy your changes.

   ./deploy 
   

Configure the Data Stream Processor to send data to a self-signed Splunk instance

The Splunk Data Stream Processor supports one-way SSL/TLS with the Splunk HEC endpoints. Use these settings if your Splunk HEC endpoints are secured via HTTPS (SSL/TLS). These settings apply globally to both Write to Index and Write to Splunk Enterprise functions. Currently, there is no support for configuring these settings outside of setting or updating environment variables before you deploy your Kubernetes pods, and you cannot change these settings on a per-connection or per-function basis.

We currently have an "all-or-nothing" approach to sending data from DSP to an SSL-enabled Splunk Enterprise instance. This means that you must have all or none of your Splunk Enterprise HEC endpoints configured to use SSL. If the K8S_PIPELINES_DATA_SPLUNKD_SSL_VALIDATION_ENABLED setting is set to true, then all DSP HEC client functions use HTTPS and require a valid certificate from the server.

Enable server certificate validation and hostname validation

1. Enable hostname verification. The DSP HEC client performs a server identity check to confirm that the client is connecting to the correct server and has not been redirected by a man-in-the-middle (MITM) attack. Defaults to false.

   ./set-config K8S_PIPELINES_DATA_SSL_HOSTNAME_VERIFICATION true
   

2. (Optional) If you are using Splunk Cloud, contact Splunk Support for a Splunk Cloud DigiCert CA.
3. (Optional) After receiving a Splunk Cloud DigiCert CA from Splunk Support or if you are using a CA cert that is not part of the Java JRE default trust store, set the following configuration value in your node.

   ./set-config K8S_PIPELINES_DATA_SPLUNKD_SSL_CERT_BASE64 [base64-encoded-CA] 

4. Enable SSL validation. When enabled, the Write to Index and Write to Splunk Enterprise functions connect to the Splunk Enterprise HEC endpoints via HTTPS and validate the server's SSL certificate. Defaults to true. Set this to true if the CA certificate you used to sign your Splunk server certificates is part of the Java JRE default trust store.

   ./set-config K8S_PIPELINES_DATA_SPLUNKD_SSL_VALIDATION_ENABLED true
   

5. After setting the configurations, deploy your changes.

   ./deploy
   

6. Restart all pipelines using the Write to Index and Write to Splunk Enterprise functions for your changes to take effect.

Disable server certificate validation and hostname validation

1. Disable hostname verification.

   ./set-config K8S_PIPELINES_DATA_SSL_HOSTNAME_VERIFICATION false
   

2. Disable SSL validation. If true, the Write to Index and Write to Splunk Enterprise functions connect to the Splunk Enterprise HEC endpoints via HTTPS and validate the server's SSL certificate. If false, the Write to Index and Write to Splunk Enterprise functions still use HTTPS but do not validate the server's SSL certificate. Defaults to true.

   ./set-config K8S_PIPELINES_DATA_SPLUNKD_SSL_VALIDATION_ENABLED false
   

3. After setting the configurations, deploy your changes.

   ./deploy
   

4. Restart all pipelines using the Write to Index and Write to Splunk Enterprise functions for your changes to take effect.

Send pipeline metrics to Splunk Enterprise for the DSP Health app

To send pipeline metrics to a Splunk Enterprise deployment via the DSP Health Application, you must set the following properties.

1. From a node in your deployment cluster, type the following.

   ./set-config K8S_FLINK_ENABLE_STATSD true
   ./set-config K8S_FLINK_STATSD_HOST <splunk-enterprise-hostname> or <splunk-enterprise-IP>
   ./set-config K8S_FLINK_STATSD_PORT 8125
   

2. After setting the configurations, deploy your changes.

   ./deploy
   

For additional setup instructions, see Install, configure, and use the DSP Health application.

Configure a TLS certificate for the Data Stream Processor UI

To configure TLS certificates for the DSP UI, set the following properties. The following instructions assume that you already have a TLS key and certificate.

1. From a node in your deployment cluster, type the following.

   base64 -w0 < tls.pem > tls.pem.b64
   base64 -w0 < tls.key > tls.key.b64
   

2. Press enter, and then type the following.

   ./set-secret K8S_NGINX_CERTIFICATE_PEM_ENCODED $(< tls.pem.b64)
   ./set-secret K8S_NGINX_CERTIFICATE_KEY_ENCODED $(< tls.key.b64)
   

3. After setting the configurations, deploy your changes.

   ./deploy