All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.
Configure your Splunk environment to monitor DSP logs
You can ingest DSP log files into your Splunk environment and use Splunk software to analyze the logs.
To get started analyzing the health of your DSP deployment with Splunk software you must configure DSP to send the log files and you must configure your Splunk environment to properly receive the log files.
The Splunk App for DSP includes a default configuration. The defaults can be adjusted to suit your deployment. The configurations may need to be distributed to other Splunk components for distributed environments.
Configure Splunk Enterprise or Splunk Cloud Platform to receive DSP log files
Before sending DSP log files to Splunk Enterprise or Splunk Cloud Platform, you must configure your Splunk environment to properly receive the log file data.
Prerequisites
You will need a Splunk instance with HEC enabled and a valid HEC token. Your HEC token must be configured to send data to an index. For information about how to enable HEC and create a HEC token, see Use the HTTP Event Collector in Splunk Web.
Make a note of the following details from your Splunk HEC configuration:
- Your Splunk Enterprise or Splunk Cloud Platform URL or IP address.
- The port assigned to Splunk HEC.
- The HTTP protocol used by Splunk HEC (HTTP or HTTPS).
- The HEC token from your Splunk HEC configuration.
- The Splunk index used to receive the log data.
Set a target index for DSP logs
You must define an index in Splunk Enterprise or Splunk Cloud Platform for your DSP log data. The default index for the DSP log data is _dsp_internal
. It is best practice for Splunk Enterprise to use the default index, but depending on your needs and local configuration, you can define a custom index in the indexes.conf
file. If you are using Splunk Cloud Platform, you must define a custom index.
See Create custom indexes for information about creating custom indexes in Splunk Enterprise. See Manage Splunk Cloud Platform indexes for information about creating indexes in Splunk Cloud Platform.
If you define a custom index, you must edit the macros.conf
file in the Splunk App for DSP and update the definition for DSP metrics index in the following stanza.
[dsp_logs_index] definition = index=_dsp_internal iseval = 0
The index defined in macros.conf
must match the target index you designate in your Splunk Enterprise or Splunk Cloud Platform configuration
To learn more about configuration files, see:
Configure DSP to send log data to the Splunk platform
Once you've configured your Splunk instance to receive DSP logs, you can configure your DSP cluster to start sending log data to the Splunk platform.
- Type the following in the working directory of the main node in your DSP cluster:
- Set the URL for your HEC-enabled Splunk environment.
dsp config set dsp-sck hec_host=<Splunk Enterprise Host IP>
- Set the port used by your HEC-enabled Splunk environment.
dsp config set dsp-sck hec_port=<your-splunk-hec-port>
- Set the HTTP protocol (HTTP or HTTPS) that your HEC-enabled Splunk environment is using.
dsp config set dsp-sck hec_protocol=<http|https>
- Set the SSL certificate verification status. Set to
false
if SSL is enabled. See Getting your certificates in the Securing Splunk Enterprise manual for more information about creating your own SSL certificates for Splunk Enterprise.dsp config set dsp-sck hec_insecure_https=<true|false>
- Set the token that your HEC-enabled Splunk environment is using.
dsp config set dsp-sck hec_token=<HEC_TOKEN>
You cannot send DSP logs to the Splunk platform without your Splunk HEC token.
- Set the log index to send log data to. If you are using a custom log index, enter the name of your custom log index instead.
dsp config set dsp-sck logs_index=<your-splunk-dsp-index>
- (Optional - Skip this step if you've already given the DSP cluster a name during installation). Give the DSP cluster a name. This name will be shown in the dashboards in the Splunk platform.
dsp config set dsp-sck cluster_name=<CLUSTER_NAME>
- Enable logs to be sent.
dsp config set dsp-sck restart_sck_logging=true
- Set the URL for your HEC-enabled Splunk environment.
- After setting the configurations, deploy your changes:
dsp deploy dsp-sck
- Wait for DSP to start sending log data to your Splunk environment.
After some time, DSP starts sending log data to the index that you defined in the [dsp_logs_index]
stanza in your Splunk platform configuration settings.
Change how DSP sends log data to the Splunk platform
You can update DSP to send log data to a different Splunk instance or change any of the related settings by re-running the commands described in the previous section. However, before your updated settings can take effect, you must disable and then re-enable log transmission.
- Disable log transmission.
dsp config set dsp-sck restart_sck_logging=false
- Deploy your updated settings.
dsp deploy dsp-sck
- Re-enable log transmission.
dsp config set dsp-sck restart_sck_logging=true
- Deploy your updated settings.
dsp deploy dsp-sck
DSP resumes sending log data to the Splunk platform, using the latest settings that you've specified.
Configure your Splunk environment to monitor DSP metrics | Use the Splunk App for DSP to monitor your DSP deployment |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6
Feedback submitted, thanks!