Splunk® Data Stream Processor

Release Notes

Acrobat logo Download manual as PDF


DSP 1.2.0 is impacted by the CVE-2021-44228 and CVE-2021-45046 security vulnerabilities from Apache Log4j. To fix these vulnerabilities, you must upgrade to DSP 1.2.4. See Upgrade the Splunk Data Stream Processor to 1.2.4 for upgrade instructions.

On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.
This documentation does not apply to the most recent version of DSP. Click here for the latest version.
Acrobat logo Download topic as PDF

Known issues for DSP

This version of the Splunk Data Stream Processor has the following known issues and workarounds.

If no issues appear here, no issues have yet been reported.

Highlighted issues

Date filed Issue number Description
2020-10-21 DSP-28545 After upgrading from DSP 1.1 to 1.2, scheduled Collect service jobs fail due to an ImagePullBackoff error.

Workaround:
Manually apply a specific service account to the service that triggers Collect service jobs. Work with Professional Services to get this service account spec.
2020-09-18 DSP-26954 Activated pipelines cannot be reactivated after upgrading.

Workaround:
Reactivate your pipelines with "Skip Restore State" enabled. Since "Skip Restore State" discards the saved state of your pipeline, your pipeline ignores any data ingested while it is deactivated causing data loss.

Uncategorized issues

Date filed Issue number Description
2021-12-03 DSP-43606 Server can be attacked by slow clients during TLS handshake
2021-09-02 DSP-41953 DSP's messaging bus does not write to a write-ahead log by default. In some cases, this may cause data loss.
2021-08-04 DSP-41040 Checkpoint cleanup for pull-based connectors cause performance issues.
2021-06-21 DSP-39621 Network policies can prevent legitimate internal service communication

Workaround:
Rarely, internal network policies can be incorrectly generated leading to disruption of internal cluster communications. Symptoms include persistent Connection Reset and Connection Timeout messages between services that should normally be allowed to communicate.

To remediate: kubectl delete netpol -A --all

If affected by this issue, internal service communication should be restored almost immediately. Note that network policy objects will be restored after running ./deploy. The above command should be re-applied after ./deploy.

2021-06-01 DSP-38591 Subseconds are not sent to Firehose
2021-05-26 DSP-38394 Multi node clusters do not work on RHEL 8.3 running on VMWare

Workaround:
A regression introduced into the RHEL 8.3 linux kernel prevents multi node DSP clusters from working properly on VMWare infrastructure when the DSP VMs are scheduled across multiple ESX hosts.

To workaround, disable checksum offloading for impacted DSP network interfaces. ethtool -K cni0 tx off ethtool -K flannel.1 tx off

Note: this must be run on every node in the cluster and every system boot. The system should recover within a few minutes.

More information about the nature of the issue is found at https://access.redhat.com/solutions/5881451

2021-02-24 DSP-34067 DSP 1.2.0 fails preflight checks on RHEL/Centos 8.3

Workaround:
Install DSP 1.2.0 on a supported OS as listed here: https://docs.splunk.com/Documentation/DSP/1.2.0/Admin/Compatibility
2021-02-23 DSP-34031 In HEC sink, unhandled exception causes pipeline restarts
2021-02-04 DSP-33028 Ingest-s2s pods in a potential deadlock state
2021-01-20 DSP-32261, DSP-28080 Pipeline fails intermittently while parsing a date format in an Apply Timestamp Extraction function
2020-12-17 DSP-31299, DSP-31300, DSP-31322 Splunk App for DSP: Improve dropdown filtering when collecting metrics from multiple clusters
2020-12-15 DSP-31160 HEC sink leaks memory when ACK is disabled
2020-12-09 DSP-30903 HEC Sink throws ArrayOutOfBoundException eventually
2020-11-20 DSP-29917 KV store lookups caching should be on by default

Workaround:
Set the cache_expiry_after_write to a positive value in milliseconds, such as 300000 (5 minutes), to set cache expiry for each value after it was loaded into the cache.
2020-11-09 DSP-29322, DSP-34095 Under low throughput conditions, records batched in batch_bytes, batch_records, and splunk_enterprise_indexes do not get flushed using the default timeout.

Workaround:
Set the millis argument in batch_bytes and batch_records or the batch_millis arg for splunk_enterprise_indexes to 8000 (8 sec) or less.


2020-10-22 DSP-28602 SignalFX sink and lookups fails to checkpoint when event has a map field.

Workaround:
The SignalFX sink functions and lookup function when using the KV Store connectors will fail to checkpoint if there is a Map in an event field. To resolve this issue ensure that events have no Map objects in any of their fields or convert any existing Map objects to an alternative type. For example using " | eval map = to_json(map)" on a field with a Map object will convert that field to a string.
2020-10-22 DSP-28649 After upgrading from 1.1 to 1.2, you cannot create new DSP HEC tokens.
2020-10-19 DSP-28395, DSP-28139 You cannot reactivate a pipeline if you do a round-trip toggle from Canvas -> SPL -> Canvas.

Workaround:
If you are editing a pipeline that has been activated before, stay in the Canvas builder.
2020-10-19 DSP-28423 The connection_id dropdown does not work for pipelines migrated from 1.1 that use the splunk_enterprise_indexes or splunk_enterprise sink functions.

Workaround:
1. Deactivate the pipeline

2. Click on the Ellipsis (More Options menu) next to "Activate Pipeline" and expand the Streams JSON Import/Export menu.

3. Update the resolvedId of the affected functions.

For splunk_enterprise_indexes, update the value of "resolvedId" to "into_splunk_enterprise_indexes:collection<record<R>>:connection-id:expression<string>:expression<string>"

For splunk_enterprise, update the value of "resolvedId" to "into_splunk_enterprise:collection<record<R>>:connection-id:expression<string>:expression<bytes>"

4. Save the updated metadata.

5. Save and reactivate the pipeline.

2020-10-14 DSP-28179, DSP-28408 After upgrading from 1.1 to 1.2, pipelines with stats functions are stuck in "RESTARTING" mode.

Workaround:
A performance improvement for the stats function prevents recovery of state. Pipelines that are stuck in "RESTARTING" need to be reactivated using the "Skip Restore State" setting in the UI (skipRestoreState in the API). Restarting with skip restore state will result in some amount of data loss.
2020-10-07 DSP-27878 Apply Line Break function metrics show twice the number of records.
2020-10-05 DSP-27716 Large batch size when using a clustered KV Store behind a GCP load balancer causes HTTPs requests to exceed allowed size (status code 431).

Workaround:
Reduce the batch size limit to make the request under 16KB.
2020-09-30 DSP-27483 On macOS, you cannot accept the warning for the self-signed TLS certificate

Workaround:
Type "thisisunsafe" in the browser window to accept the certificate.
2020-09-18 DSP-26912 Generated SPL can erroneously duplicate statement names when statement names of the form statement_n are hard-coded in the Pipeline JSON.

Workaround:
Change the hard-coded statement names in the Pipeline JSON to something that does not match statement_n.
2020-09-10 DSP-26551 DSP installation pre-flight check for disk space should check the correct volume/directory.

Workaround:
/var/data is always erroneously checked for disk size, even if another volume is specified with --location. You can symlink where you intend to install to /var/data as a workaround.

For example, if you use --location /data, first ln -s /data /var/data

2020-08-31 DSP-26064 Upgrading DSP gives a "HTTPS Status 500 - Internal Server Error" on initial login.

Workaround:
Clear your browser cache and then try logging back into DSP.
2020-08-17 DSP-25339 DSP does not run on an operating system that has FIPS mode enabled.
2020-04-30 DSP-20769 Migration prompt continues to show up after an active pipeline is migrated to a new API version.
2020-04-21 DSP-20380 When toggling between the Canvas Builder and SPL2 Builder, function configurations are unexpectedly populated with default values.

Workaround:
Configure every function so the UPL is valid.
2020-04-21 DSP-20379 User named functions are lost when toggling between the Canvas and the SPL2 Builders.
2020-04-08 DSP-19759 DSP does not properly ingest sourcetypes with INDEXED_EXTRACTIONS=true.
Last modified on 18 March, 2022
PREVIOUS
New features for DSP
  NEXT
Fixed Issues for DSP

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters