Splunk® Data Stream Processor

Release Notes

Acrobat logo Download manual as PDF


DSP 1.2.0 is impacted by the CVE-2021-44228 and CVE-2021-45046 security vulnerabilities from Apache Log4j. To fix these vulnerabilities, you must upgrade to DSP 1.2.4. See Upgrade the Splunk Data Stream Processor to 1.2.4 for upgrade instructions.

On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.
This documentation does not apply to the most recent version of Splunk® Data Stream Processor. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

New features for DSP

Here's what's new in each version of the Splunk Data Stream Processor

Version 1.2.0

What's new in the docs

The DSP documentation was refactored in version 1.2.0 to present information in a more intuitive manner and better reflect the end-to-end user experience of the product. As a result, the titles and locations of some topics have changed.

  • The Getting Data In manual has been replaced by the Connecting to Data Sources and Destinations manual, which provides complete information about how to connect your DSP pipeline to a given data source or data destination.
  • The contents of the Use the Data Stream Processor manual have been reorganized.
  • The contents of the following chapters from the Use the Data Stream Processor manual have been moved into the new Connecting to Data Sources and Destinations manual:
    • "Pipeline requirements for specific data sources in DSP"
    • "Format data in DSP to send to the Splunk platform"
    • "Send data from DSP to other destinations"
  • The Source functions (Data Sources) and Sink functions (Data Destinations) topics in the Function Reference manual have been moved and rewritten. The source and sink functions are now in dedicated chapters of the Function Reference manual.
  • The Send data from Splunk DSP to SignalFx topic has been rewritten and now includes a detailed example demonstrating how to send metrics.log data from the Splunk universal forwarder to SignalFx.
  • Added TLS/Cipher suite information.
  • Updated the DSP HEC examples, and added documentation about multi-metric support.

What's new in the Data Stream Processor

The following table describes new features or enhancements in DSP 1.2.0.

New Feature or Enhancement Description
Support for CSV and Splunk Enterprise KV Store lookups DSP now supports lookups to Splunk Enterprise KV Stores or CSV files for increased data enrichment.


See About lookups and Lookup for more information.

Support for sending data to a Splunk Enterprise KV Store collection DSP now supports writing data from DSP into a Splunk Enterprise KV Store collection.


See Connect Splunk DSP to a Splunk Enterprise KV Store and Write Thru KV Store for more information.

Streaming ML Streaming ML is the Splunk Enterprise machine learning framework designed specifically for online learning. This framework includes a library of operators that enable users to apply machine learning models to streaming data, without requiring offline batch training jobs. Steaming ML in DSP 1.2 includes three new functions for Time Series Decomposition (STL), Pairwise Categorical Outlier Detection, Percentiles, and more. You must install the Streaming ML plugin to access these functions. All Streaming ML functions are currently in beta.


For more information, see About the Streaming ML Plugin.

Apply Line Break You can now perform line breaking and merging for universal forwarder data in one function. In addition, you can now migrate and reuse existing props.conf line_breaking configurations in DSP.


See Apply Line Break for more information.

Apply Timestamp Extraction You can now extract additional timestamp formats using strptime() and regular expressions. In addition, you can now migrate and reuse existing props.conf timestamp extraction configurations in DSP.


See Apply Timestamp Extraction for more information.

Apache Pulsar Connector DSP now supports collecting data from an Apache Pulsar topic.


See Connecting Apache Pulsar to your DSP pipeline as a data source and Get data from Apache Pulsar for more information.

Google Cloud Pub/Sub Connector DSP now supports collecting messages from Google Cloud Pub/Sub.


See Connecting Google Cloud Pub/Sub to your DSP pipeline and Get data from Google Cloud Pub/Sub for more information.

Send to SignalFx (trace) You can now send trace data to a SignalFx endpoint using the SignalFx connector.


See Connecting SignalFx to your DSP pipeline and Send data to SignalFx (trace) for more information.

Updates to the Splunk App for DSP The DSP Health application has been renamed to Splunk App for DSP.

You can now collect additional metrics about your DSP environment and monitor those metrics in Splunk Enterprise. In addition, there are now more dashboards to help you visualize the health of your DSP environment.


See About the Splunk App for DSP for more information.

New install flavors and profiles. DSP now supports additional install flavors and node roles. In addition, DSP also supports more than five master nodes in a cluster.
Updated Send to Microsoft Azure Event Hubs sink function This sink function now provides improved performance and data batching controls.


See Send data to Microsoft Azure Event Hubs (Beta) for more information.

Updated Send to Amazon S3 sink function You can now compress the data that you send to Amazon S3. When sending data in Parquet format, you can now specify the version of Parquet Writer to use, the maximum size of each row group, and how DSP handles records with invalid schemas.

Files generated by this function are now given the correct filename extension based on the file format.


See Send data to Amazon S3 for more information.

SPL2 Named Arguments DSP now supports named arguments when using SPL2 (Search Processing Language version 2) for source, sink, and scalar functions.


See the SPL2 examples in the Function Reference manual for more information.

Dot and bracket notation support for accessing lists and maps It's now easier to access list and maps.


See Accessing list elements using bracket notation and Accessing map elements using dot notation.

map_merge scalar function You can now merge two or more maps together in DSP using the map_merge scalar function.


See map_merge for more information.

Improved performance of the Forwarders Service Changes to the Forwarders Service for better performance.
Updated names for connectors and functions The display names that appear in the DSP UI for connectors, source functions, and sink functions have been updated for clarity and consistency. Additionally, the SPL2 names for some functions have been updated. See the "Renamed functions in version 1.2.0" section on this page for more information.
SCloud 4 SCloud 4.0 is now bundled with DSP.
--location install flag You can now specify a location for Gravity to save container and state information using a --location flag.

What's new in the DSP SDK

The following table describes new features or enhancements in the DSP SDK.

New Feature or Enhancement Description
RuntimeContext#getArgument() no longer replaces dashes in argument names with underscores Previously, scalar function arguments could be accessed from RuntimeContext using dash-cased argument names. Now, all argument names must be accessed using their underscore_cased names.
Record#get() returns read-only view of maps and lists Previously, functions were able to read maps or lists from Record and directly modify them. Now, maps or lists read from Record must be explicitly copied before they can be modified.
AggregationFunction#initialState() is deprecated Update classes that implement AggregationFunction to use AggregationFunction#initialState(RuntimeContext) instead.

Renamed SPL2 functions in version 1.2.0

The following functions were renamed in 1.2.0.

Original SPL2 function name Updated SPL2 function name
read_event_hubs event_hubs
read_kafka kafka
read_kinesis kinesis
read_splunk_firehose splunk_firehose
receive_from_forwarders forwarders
receive_from_ingest_rest_api ingest_rest_api
write_index index
write_kafka kafka
write_kinesis kinesis
write_null dev_null

Version 1.1.0

New Feature or Enhancement Description Learn more link
SPL2 Support DSP now supports creating and configuring DSP pipelines using SPL2 (Search Processing Language version 2). SPL2 for DSP.
SPL2 Builder DSP now supports an additional pipeline builder experience allowing you to write pipelines in SPL2. SPL2 Pipeline Builder.
DSP HTTP Event Collector You can send events and metrics to a DSP data pipeline using the DSP HTTP Event Collector (DSP HEC). The DSP HEC supports the Splunk HTTP Event Collector (HEC) /services/collector, /services/collector/event, and /services/collector/event/1.0 endpoints allowing you to quickly redirect your existing Splunk HEC workflow into DSP via the DSP Firehose. Send events to a DSP data pipeline using the DSP HTTP Event Collector.
Syslog support You can now easily ingest syslog data into DSP using Splunk Connect for Syslog (SC4S). Send Syslog events to a DSP data pipeline using SC4S with DSP HEC.
Amazon Linux 2 support DSP now supports Amazon Linux 2. Hardware and Software requirements.
Upgraded Streams REST API Upgraded Streams REST API endpoints to v3beta1 Splunk Data Stream Processor REST API Reference.
Apache Pulsar messaging bus DSP now uses Apache Pulsar as its messaging bus for data sent via the Ingest, Collect, and Forwarders Services. Increase Pulsar partitions for improved pipeline throughput
Splunk Enterprise sink function with Batching You can now do index-based routing even while batching records. This function performs the common workflow of mapping the DSP event schema to Splunk HEC metrics or events schema, turning records into JSON payloads, and batching the bytes of those payloads for better throughput. Write to the Splunk platform with Batching
Splunk Enterprise sink function This function replaces Write Splunk Enterprise. This function adds out of the box support for index-based routing while batching. Write to the Splunk platform
Batch Bytes streaming function DSP now supports batching your data as byte payloads for increased throughput. Batch Bytes
To Splunk JSON streaming function You can now perform automatic mapping of DSP events schema to Splunk HEC events or metrics schema. To Splunk JSON.
Write to S3-compatible storage sink function DSP now supports sending data to an Amazon S3 bucket. Write to S3-compatible storage
Write to SignalFx sink function DSP now supports sending data to a SignalFx Endpoint. Write to SignalFx
Microsoft 365 Connector DSP now supports collecting data from Microsoft 365 and Office 365 services using the Microsoft 365 Connector. Use the Microsoft 365 Connector with Splunk DSP.
Google Cloud Monitoring Metrics Connector DSP now supports collecting metrics data from Google Cloud Monitoring. Use the Google Cloud Monitoring Metrics Connector with Splunk DSP.
Amazon S3 Connector The Amazon S3 Connector now supports Parquet format as a File Type. Use the Amazon S3 Connector with Splunk DSP.
Write to Azure Event Hubs Using SAS Key sink function (Beta) DSP now supports sending data to an Azure Event Hubs namespace using an SAS key. This is a beta function and not ready for production. Write to Azure Event Hubs.
Bug fixes The Splunk Data Stream Processor 1.1.0 includes several bug fixes. Fixed Issues for DSP.

Version 1.0.1

  • Bug fixes. For details, see Fixed issues.

Version 1.0.0

This is the first release of the Splunk Data Stream Processor.

Last modified on 19 February, 2021
  NEXT
Known issues for DSP

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters