Splunk® Data Stream Processor

Install and administer the Data Stream Processor

On April 3, 2023, Splunk Data Stream Processor reached its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.

All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.

Get started with the Splunk Cloud Services CLI

The includes the Splunk Cloud Services (SCloud) command-line tool making it easy to make API calls directly to DSP services. You can use scloud to interact with APIs such as the Ingest service.

Starting in 1.2.0, the includes both the Splunk Cloud Services CLI 1.0.0 and Splunk Cloud Services CLI 4.0.0. By default, the Splunk Cloud Services CLI 4.0.0 is used when running the base ./scloud command. The table below details the main differences between the two versions, but there are several other differences between Splunk Cloud Services CLI 1.0.0 and Splunk Cloud Services CLI 4.0.0. For the most comprehensive Splunk Cloud Services CLI help, use the ./scloud help command for the desired Splunk Cloud Services CLI version. Best practices are to use Splunk Cloud Services CLI 4.0.0, as it supports the latest DSP features, but 1.0.0 can still be used with many of the existing APIs.

Splunk Cloud Services CLI 1.0.0 Splunk Cloud Services CLI 4.0.0
Global configurations are saved in an ~/.scloud file Global configurations are saved in an ~/.scloud.toml file.
Flags are set using a single hyphen -, for example ./scloud ingest post-events -format raw <<< 'test' Most flags are set using a double hyphen --, for example ./scloud ingest post-events --format raw <<< 'test'
Settings that accept boolean values must be quoted. See the insecure setting in the "If you want to use SCloud 1.0.0" section for an example. Settings that accept boolean values cannot be quoted. See the insecure setting in the "If you want to use SCloud 4.0.0" section for an example.
Use ./scloud forwarders create-certificate my_forwarder.pem to format and upload the forwarders certificate. Use ./scloud forwarders add-certificate --input-datafile my_forwarder.pem to format and upload the forwarders certificate.
./scloud streams uses the v2beta1 endpoints. ./scloud streams uses the v3beta1 endpoints.
./scloud login returns the authorization token you'll need to cURL DSP endpoints. ./scloud login --verbose returns the authorization token you'll need to cURL DSP endpoints.
./scloud get-context-cache prints the bearer token for the current SCloud session. ./scloud context list prints out the bearer token for the current SCloud session.
Not applicable. ./scloud ingest contains the new DSPHEC endpoints.
Not applicable. You can now perform CRUD operations on jobs with the ./scloud collect endpoints.

Configure the Splunk Cloud Services CLI

How you configure the Splunk Cloud Services CLI varies depending on what version of the Splunk Cloud Services CLI you are using. Best practices are to use the Splunk Cloud Services CLI 4.0.0, as it supports the latest DSP features, but the Splunk Cloud Services CLI 1.0.0 can still be used with many of the existing APIs.

All of the Splunk Cloud Services CLI documentation examples for DSP 1.2.0 use the latest version of the Splunk Cloud Services CLI.

See Configure SCloud in version 1.1.0 of the Install and administer the Data Stream Processor guide if you want to configure and use the Splunk Cloud Services CLI 1.0.0 with DSP 1.2.0. If you want to use the Splunk Cloud Services CLI 1.0.0, make sure you either explicitly use scloud.v1 for each Splunk Cloud Services CLI command, or update the scloud symlink to point to scloud.v1.

Follow these steps to configure the Splunk Cloud Services CLI 4.0.0.

  1. From the DSP directory of the controller node, set DSP_HOST to that node's IP address.
    DSP_HOST=<IP_Address>
  2. Run the following script by copy and pasting the following text in the command line.
    cat << EOF > ~/.scloud.toml
    env = "prod"
    host-url = "https://$DSP_HOST"
    auth-url = "https://$DSP_HOST"
    tenant = "default"
    username = "dsp-admin"
    insecure = true
    EOF
    

    Settings that accept boolean values no longer need to be enclosed in quotes in the Splunk Cloud Services CLI 4.0.0. Make sure that insecure = true does not have surrounding quotes.

Verify that the Splunk Cloud Services CLI tool has been configured properly

Follow these steps to verify that the Splunk Cloud Services CLI has been configured successfully for your DSP environment.

  1. From the DSP installer directory, login to the Splunk Cloud Services CLI.
    ./scloud login

    When prompted, use the dsp-admin password printed during the DSP installation. If needed, you can display the default password by typing sudo ./dsp admin print-login on one of your nodes.

    The Splunk Cloud Services CLI doesn't return your login metadata or access token. If you want to see your access token you must log in to the Splunk Cloud Services CLI using the verbose flag: ./scloud login --verbose.

  2. (Optional) From the DSP homepage, click on Get Started and select the Splunk DSP Firehose source.
  3. (Optional) Click the Start Preview Start Preview button button.
  4. Try ingesting an event.
    ./scloud ingest post-events --format raw <<< 'test'

Update your Splunk Cloud Services CLI password

You can change the dsp-admin password that you use to login to the Splunk Cloud Services CLI and the DSP UI. This script does not update the password for other users.

  1. From the working directory of a controller node, run the reset password script.
    sudo dsp admin reset-password
  2. Enter your new password.
  3. Log in to the Splunk Cloud Services CLI with your new password.
    ./scloud login

Additional commands

The syntax for the Splunk Cloud Services CLI is as follows:
./scloud [-u <username>] [-p <password>] [-tenant <tenant>] [-no-prompt]<command> <subcommand> [params]

scloud has the following additional commands. For subcommands and more information, run ./scloud help or ./scloud <command> help.

The ./scloud help command returns a list of services, some of which are not compatible with the Data Stream Processor. The services detailed below are the services specific to the Data Stream Processor.

Service Description
Identity The identity commands create and manage users and admins, manage tenant memberships, and manage the access to tenant resources granted by the definition of groups, roles, and their corresponding permissions.
Streams The streams command creates and manages pipelines, connections, preview sessions, and other CRUD operations associated with your DSP pipelines.
Forwarders The forwarders command interacts with the Splunk forwarder service APIs to create, delete, and view certificates that the forwarders use. It does not control forwarders.
Ingest The ingest command posts event and metric data to the ingest service.
Collect The collect command manages data collection jobs.
Last modified on 13 January, 2023
Manage users and admins   Get started with the Data Stream Processor CLI

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters