Splunk® Data Stream Processor

Use the Data Stream Processor

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Back up, restore, and share pipelines using SPL2

You can back up a pipeline by saving the SPL2 associated with the pipeline. Currently, there is no pipeline versioning mechanism, so you may find it helpful to manually back up and save your pipelines in a version control system in case something unexpected happens.

Create a backup of a pipeline

Follow these steps to create a backup of a pipeline.

  1. From the home page, click Data Management and find the pipeline that you want to save a copy of.
  2. Open the pipeline, and select one of the following steps. If you select an active pipeline, click Edit before continuing.
    • Click the three dots in the pipeline row, and click Clone. Make any modifications that you want to the cloned pipeline, and maintain the original pipeline as a precautionary measure.
    • Click SPL next to the pipeline name to toggle to the SPL2 builder. Copy the SPL2 for your pipeline and save it to your preferred location for storing backups.

      If you selected a previously active pipeline, do not save this pipeline after toggling to the SPL2 builder. Instead, save the SPL2 for this pipeline in your preferred location, and close the pipeline without saving your changes. There is a known issue that can cause data loss when you use the toggle on previously active pipelines. See the known issues page.

You now have a backup of a pipeline in case something goes wrong with your pipeline.

Restore a pipeline using SPL2 in the UI

Follow these steps to restore a pipeline using SPL2 in the UI.

  1. From the home page, click Data Management > Create New Pipeline.
  2. Select the SPL2 Builder.
  3. Paste the SPL2 of your desired pipeline into the SPL2 Pipeline Builder box. For an example, try pasting the following SPL2 into the SPL2 Pipeline Builder box.
    | from splunk_firehose() | fields - nanos, id | into dev_null();
    
  4. Click Build pipeline
  5. Click Save.
  6. Give your pipeline a name, and click Save again.

Backup and restore a pipeline using the Splunk Cloud Services CLI

You can back up and restore a pipeline using the Splunk Cloud Services CLI. Use the CLI to export the Streams JSON of the pipeline, and then recreate the pipeline based on the exported Streams JSON. The Streams JSON is an abstract representation of a data pipeline using JSON syntax.

The following steps assume that you have jq installed.

  1. From the command line, log in to the Splunk Cloud Services CLI.
    ./scloud login
    
  2. To back up a pipeline by exporting the corresponding Streams JSON, do the following:
    1. List the pipelines that are currently in your tenant.
      ./scloud streams list-pipelines
    2. Find the pipeline that you want to create, and copy its id.
    3. Export the Streams JSON of the pipeline. The following command saves the underlying Streams JSON associated with your pipeline in a designated JSON file.
      ./scloud streams get-pipeline --id <ID> | jq .data > <pipeline-name>.json
  3. To restore a pipeline by recreating it based on the Streams JSON, run the following command.
    ./scloud streams create-pipeline --name "<pipeline-name>" --input-datafile <path-to-json-file> --bypass-validation true
Last modified on 04 February, 2021
PREVIOUS
Send data from a pipeline to multiple destinations
  NEXT
Using activation checkpoints to activate your pipeline

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters