Splunk® Data Stream Processor

Use the Data Stream Processor

Acrobat logo Download manual as PDF


On April 3, 2023, Splunk Data Stream Processor reached its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.

All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.
Acrobat logo Download topic as PDF

Create a pipeline using the Canvas View

The Canvas View provides a graphical user interface to incrementally build a pipeline. You can then configure your pipeline using SPL2 (Search Processing Language) expressions. Use the Canvas View if you want:

  • Assistance building a pipeline using UIs to configure each of your functions.
  • To create a pipeline with many branches, and you want a more visual representation of this pipeline.

The pipeline toggle button is currently a beta feature and repeated toggles between the two views can lead to unexpected results. If you are editing an active pipeline, using the toggle button can lead to data loss.

Create a new pipeline

Follow these steps to create a new pipeline.

  1. From the home page, click Get Started.
  2. Select a data source.
    The Canvas View opens.
  3. Give your pipeline a name by clicking the pencil button Splunk Data Stream Processor rename button. You can additionally give your pipeline a description by clicking the pipeline options button DSP Ellipses button and then selecting Update Pipeline Metadata.
  4. Configure your source function. If you are using a connection-based function, create a connection before using this function. See Getting started with data connections in the Connect to Data Sources and Destinations with the manual.
  5. Click the + icon to add a function.
    A navigation bar on the right appears.
  6. Select a function from the navigation bar. For a full list of all functions available, see DSP functions by category.
  7. Configure the function.
  8. End your pipeline with a sink function.
  9. To validate your pipeline, click the pipeline options button DSP Ellipses button and then select Validate.
    If you have any invalid functions, the first invalid function in your pipeline is highlighted in red and all subsequent functions are in an unsure state denoted by a question mark.
  10. Click Save.
  11. (Optional) Click Activate to activate your pipeline.
  12. Click Pipelines to go to the Pipelines page. After saving your pipeline, use the Pipelines page to edit, delete, clone, or activate your pipeline.

Edit a pipeline

Follow these steps to edit a pipeline.

  1. Click the Pipelines tab to go to the Pipelines page.
  2. Click the name of a pipeline to view it on the Canvas View.
  3. If the pipeline is currently active, click Edit to be taken to a copy of your active pipeline. Any changes you make are made on that copy and don't affect the active version of your pipeline. If the pipeline is currently inactive, any changes are made directly to your pipeline.
  4. Click Save to save changes to your pipeline.
  5. (Optional) Click Activate or Activate with changes to activate your pipeline. If you were working on a copy of an active pipeline, this updates the version of your current active pipeline to the latest version.
    If you are attempting to re-activate your pipeline and running into issues, you may want to update your activation checkpoint. See Using activation checkpoints to activate your pipeline.
  6. Click the Pipelines tab to return to the Pipelines page.
    After saving your pipeline, use the Pipelines page to edit, delete, clone, or activate your pipeline.
  7. (Optional) If you were editing a pipeline that is currently active, in the pipelines listing table, click the options button DSP Ellipses button beside your pipeline and select Upgrade to latest. This updates your pipeline to the latest version. This may impact performance, but no data is lost during this process.

Differences and inconsistent behaviors between the Canvas View and the SPL View

You can toggle between the Canvas View and the SPL View. There are several differences between the Canvas View and the SPL View.

Differences

Canvas View SPL View
Accepts SPL2 Expressions as input. Accepts SPL2 Statements as input. SPL2 statements support variable assignments and must be terminated by a semi-colon.
For source and sink functions, optional arguments that are blank are automatically filled in with their default values. If you want to specify any optional arguments in source or sink functions, you must provide all arguments up until the optional argument that you want to omit, and you must omit all subsequent optional arguments.
When using a connector, the connection name can be selected as the connection-id argument from a dropdown in the UI. When using a connector, the connection-id must be explicitly passed as an argument. You can view the connection-id of your connections by going to the Connections Management page.
Source and sink functions have implied from and into functions respectively. Source and sink functions require explicit from and into functions.

Toggling between the SPL2 View and the Canvas View (BETA)

The toggle button is currently a beta feature. Repeated toggles between the Canvas View and the SPL2 View may produce unexpected results. The following table describes specific use cases where unexpected results have been observed.

If you are editing an active pipeline, toggling between the Canvas and the SPL views can lead to data loss as the pipeline state is unable to be restored. Do not use the pipeline toggle if you are editing an active pipeline.

Pipeline contains... Behavior
Variable assignments Variable names are changed and statements may be reordered.
Renamed function names Functions are reverted back to their default names.
Comments Comments are stripped out when toggling between views.
A stats or aggregate with trigger function that uses an evaluation scalar function within the aggregation function. Each function is called separately when toggling from the SPL2 View to the Canvas View. For example, if your stats function contained the if scalar function as part of the aggregation function count(if(status="404", true, null)) AS status_code, the Canvas View assumes that you are calling two different aggregation functions: count(if(status="404") and if(status="404") AS status_code both of which are invalid.
Last modified on 23 March, 2022
PREVIOUS
Navigating the
  NEXT
Create a pipeline using the SPL View

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters