Splunk® Data Stream Processor

Install and administer the Data Stream Processor

On April 3, 2023, Splunk Data Stream Processor will reach its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.
This documentation does not apply to the most recent version of Splunk® Data Stream Processor. For documentation on the most recent version, go to the latest release.

Get started with SCloud

SCloud is a command-line tool to make API calls to the Data Stream Processor. You can use SCloud to interact with APIs such as the Ingest REST API. DSP 1.1.0 comes with SCloud 1.0.0. To use all the new features of DSP 1.1.0, upgrade to SCloud 4.0.0.

Upgrade from SCloud 1.0.0 to SCloud 4.0.0

Best practices are to use SCloud 4.0.0, as it supports the latest DSP features, but SCloud 1.0.0 can still be used with many of the existing APIs. You only need to upgrade SCloud to version 4.0.0 to interact with the new v3beta1 Streams API, the Collect Service, or the new Ingest /collector endpoints. If you don't want to interact with these services, you may continue using SCloud 1.0.0.


  1. To download SCloud 4.0.0, open your command-line interface and run one of the following commands:
    • If you use cURL:
    curl -L -o 'scloud_v4.0.0_linux_amd64.tar.gz' 'https://github.com/splunk/splunk-cloud-sdk-go/releases/download/v1.7.0/scloud_v4.0.0_linux_amd64.tar.gz'
    • If you use Wget:
    wget 'https://github.com/splunk/splunk-cloud-sdk-go/releases/download/v1.7.0/scloud_v4.0.0_linux_amd64.tar.gz'
  2. Extract the SCloud 4.0.0 file you just download by running this command: 
    tar -xvzf scloud_v4.0.0_linux_amd64.tar.gz
  3. Reconfigure SCloud to point to your DSP environment. See "Configure SCloud" on this page for more information.

SCloud 4.0.0 introduces the following changes.

SCloud 1.0.0 SCloud 4.0.0
Global configurations are saved in an ~/.scloud file Global configurations are saved in an ~/.scloud.toml file.
Flags are set using a single hyphen -, for example ./scloud ingest post-events -format raw <<< 'test' Most flags are set using a double hyphen --, for example ./scloud ingest post-events --format raw <<< 'test'
Settings that accept boolean values must be quoted. See the insecure setting in the "Configure SCloud" section for an example. Settings that accept boolean values cannot be quoted. See the insecure setting in the "Configure SCloud" section for an example.
Use ./scloud forwarders create-certificate my_forwarder.pem to format and upload the forwarders certificate. Use ./scloud forwarders add-certificate --input-datafile my_forwarder.pem to format and upload the forwarders certificate.
./scloud streams uses the v2beta1 endpoints. ./scloud streams uses the v3beta1 endpoints.
./scloud login returns the authorization token you'll need to cURL DSP endpoints. ./scloud login --verbose returns the authorization token you'll need to cURL DSP endpoints.
./scloud get-context-cache prints the bearer token for the current SCloud session. ./scloud context list prints out the bearer token for the current SCloud session.
Not applicable. ./scloud ingest contains the new DSPHEC endpoints.
Not applicable. You can now perform CRUD operations on jobs with the ./scloud collect endpoints.

Configure SCloud

How you configure SCloud varies depending on what version of SCloud you are using. Follow the instructions in the table for the version of SCloud that you have. If you have the SCloud version that comes pre-packaged with DSP, you have SCloud 1.0.0. You can double-check what version of SCloud you have by running scloud version.

If necessary you can update the SCloud version that is launched by changing the symlink that scloud points to. For example, if you are using SCloud 4.0.0 and you want to change to SCloud 1.0.0, type: ln -sf scloud.v1 scloud.

SCloud 1.0.0 SCloud 4.0.0

1. Run the following command from the DSP directory of your master node to set DSP_HOST to the IP address of your master node: 

DSP_HOST=<IP_Address>

2. Generate a configuration file in the home directory by running the following script in the command line: 

cat << EOF > ~/.scloud
env = "prod"
host-url = "https://$DSP_HOST:31000"
auth-url = "https://$DSP_HOST:31000"
tenant = "default"
username = "dsp-admin"
insecure = "true"
EOF

1. Run the following command from the DSP directory of your master node to set DSP_HOST to the IP address of your master node: 

DSP_HOST=<IP_Address>

2. Generate a configuration file in the home directory by running the following script in the command line: 

cat << EOF > ~/.scloud.toml
env = "prod"
host-url = "https://$DSP_HOST:31000"
auth-url = "https://$DSP_HOST:31000"
tenant = "default"
username = "dsp-admin"
insecure = true
EOF


Verify that the SCloud tool has been configured properly

  1. Before you execute the following commands, navigate to your DSP directory where you've just installed and configured SCloud using the command-line interface.
  2. From the command line, type: 
    ./scloud login
    When prompted, use the dsp-admin password printed during the DSP installation. If needed, you can get the username and password again by typing sudo ./print-login on one of your nodes.
  3. From the DSP homepage, click on Build Pipeline and select the Splunk Firehose to Splunk Index template.
  4. Click Start Preview.
  5. Try ingesting an event.
    • If you are using SCloud 1.0.0:
    ./scloud ingest post-events -format raw <<<  'test'
    • If you are using SCloud 4.0.0:
    ./scloud ingest post-events --format raw <<<  'test'

Update your SCloud password

You can change the dsp-admin password that you use to login to SCloud and the DSP UI. This script does not update the password for other users.

  1. From the master node, run the reset password script. 
    sudo ./reset-admin-password
  2. Enter your new password.
  3. Log in to SCloud with your new password. 
    ./scloud login


Additional commands

The syntax for the Splunk Cloud CLI is as follows:
./scloud [-u <username>] [-p <password>] [-tenant <tenant>] [-no-prompt]<command> <subcommand> [params]

scloud has the following additional commands. For subcommands and more information, run ./scloud help or ./scloud <command> help.

Service Description
Identity The identity commands create and manage users and admins, manage tenant memberships, and manage the access to tenant resources granted by the definition of groups, roles, and their corresponding permissions.
Streams The streams command creates and manages Splunk data stream pipelines.
Forwarders The forwarders command interacts with the Splunk forwarder service APIs to create, delete, and view certificates that the forwarders use. It does not control forwarders.
Ingest The ingest command posts event and metric data to the ingest service.
Collect The collect command manages data collection jobs.
Last modified on 23 October, 2020
Manage users and admins   Cluster configuration options

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.1.0

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters