Back up, restore, and share pipelines using SPL2

You can back up a pipeline by saving the SPL2 associated with the pipeline. Currently, there is no pipeline versioning mechanism, so you may find it helpful to manually back up and save your pipelines in a version control system in case something unexpected happens.

Create a backup of a pipeline

Follow these steps to create a backup of a pipeline.

1. From the home page, click Data Management and find the pipeline that you want to save a copy of.
2. Open the pipeline, and select one of the following steps. If you select an active pipeline, click Edit before continuing.
   • Click the three dots in the pipeline row, and click Clone. Make any modifications that you want to the cloned pipeline, and maintain the original pipeline as a precautionary measure.
   • Click SPL next to the pipeline name to toggle to the SPL2 builder. Copy the SPL2 for your pipeline and save it to your preferred location for storing backups.

     If you selected a previously active pipeline, do not save this pipeline after toggling to the SPL2 builder. Instead, save the SPL2 for this pipeline in your preferred location, and close the pipeline without saving your changes. There is a known issue that can cause data loss when you use the toggle on previously active pipelines. See the known issues page.

You now have a backup of a pipeline in case something goes wrong with your pipeline.

Restore a pipeline using SPL2 in the UI

Follow these steps to restore a pipeline using SPL2 in the UI.

1. From the home page, click Data Management > Create New Pipeline.
2. Select the SPL2 Builder.
3. Paste the SPL2 of your desired pipeline into the SPL2 Pipeline Builder box. For an example, try pasting the following SPL2 into the SPL2 Pipeline Builder box.

   | from splunk_firehose() | fields - nanos, id | into dev_null();
   

4. Click Build pipeline
5. Click Save.
6. Give your pipeline a name, and click Save again.

Backup and restore a pipeline using the Splunk Cloud Services CLI

You can back up and restore a pipeline using the Splunk Cloud Services CLI. Use the CLI to export the Streams JSON of the pipeline, and then recreate the pipeline based on the exported Streams JSON. The Streams JSON is an abstract representation of a data pipeline using JSON syntax.

The following steps assume that you have jq installed.

1. From the command line, log in to the Splunk Cloud Services CLI.

   ./scloud login
   

2. To back up a pipeline by exporting the corresponding Streams JSON, do the following:
   1. List the pipelines that are currently in your tenant.

      ./scloud streams list-pipelines

   2. Find the pipeline that you want to create, and copy its id.
   3. Export the Streams JSON of the pipeline. The following command saves the underlying Streams JSON associated with your pipeline in a designated JSON file.

      ./scloud streams get-pipeline --id <ID> | jq .data > <pipeline-name>.json

3. To restore a pipeline by recreating it based on the Streams JSON, run the following command.

   ./scloud streams create-pipeline --name "<pipeline-name>" --input-datafile <path-to-json-file> --bypass-validation true