Splunk® Data Stream Processor

Use the Data Stream Processor

On April 3, 2023, Splunk Data Stream Processor will reach its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.
This documentation does not apply to the most recent version of Splunk® Data Stream Processor. For documentation on the most recent version, go to the latest release.

Using activation checkpoints to activate your pipeline

When a data pipeline is first created, it is in a deactivated state. You must activate a data pipeline to start ingesting data. When a data pipeline is activated for the first time, it starts with no state. Periodically, a checkpoint is added. The checkpoint is used if the data pipeline fails or is restarted. This allows the data pipeline to restart from the point it left off. By default, when a data pipeline is deactivated, the progress state is saved and used the next time the data pipeline is activated. See the Streams API for more information.

Data pipeline activation options

There are three different type of activation options. If you are activating your pipeline for the first time, simply chose Activate in the UI with no other settings enabled.

Activate
In some cases, you may not want to use the latest version of a data pipeline. Set activateLatestVersion = false to use the current version of the data pipeline and ignore the latest version. You can only change this setting using SCloud or the Streams API.
Default: true
Skip Restore State
If you don't want to resume a data pipeline from the latest savepoint, click on Activate Pipeline with Changes from the pipeline canvas view and click on Skip Restore State. Warning: If enabled, your pipeline ignores any data ingested while it is deactivated. If you are activating a pipeline for the first time, you don't need to enable this setting.
Default: false
Allow Non Restored State
In some cases, when you change a data pipeline, the progress state is stored for functions that no longer exist. By default, a data pipeline fails when it is activated while it has an unused progress state. Set allowNonRestoredState = true to allow the data pipeline to ignore any unused progress states, and activate your pipeline without verifying function checkpoints. To enable this setting in the UI, click on Activate Pipeline with Changes from the pipeline canvas view and click on Allow Non Restored State. If skipRestoreState is set to true, this parameter is ignored. If you are activating a pipeline for the first time, you don't need to enable this setting.
Default: false

Setting allowNonRestoredState or skipRestoreState to true can lead to data loss.

Pipeline deactivation options

skipSavepoint
Set skipSavepoint = true to skip saving the data pipeline state when it is deactivated. You can only enable this setting in SCloud or using the Streams API.
Default: false
Last modified on 28 September, 2020
Data Stream Processor data types   About Splunk Data Stream Processor regular expressions

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.1.0

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters