Splunk® Data Stream Processor

Install and administer the Data Stream Processor

Download manual as PDF

Download topic as PDF

Install the Data Stream Processor

To install the Splunk Data Stream Processor, download, extract, and run the installer on each node in your cluster. You must contact your Splunk representative to access the Data Stream Processor download page. The Data Stream Processor is installed from a Gravity package, which spins up a group of cloud instances and hardware into a cluster and deploys Kubernetes on top of it. All Data Stream Processor services gets deployed on top of Kubernetes, see the Gravity documentation for more information.

Extract and run the Data Stream Processor installer

Do the following steps to extract and run the DSP installer.


  • Before you install the Splunk Data Stream Processor, make sure that your system clocks are synchronized on each node. Consult the system documentation for the particular operating systems on which you are running the Splunk Data Stream Processor. For most environments, Network Time Protocol (NTP) is the best approach.
  • You must have system administrator (root) permissions to run DSP installation or cluster management commands. If you do not have these permissions, you can use the sudo command.


  1. Download the Data Stream Processor installer on each node in your cluster.
  2. On each node in your cluster, extract the Data Stream Processor installer from the tarball.
    tar xf <dsp-version>-linux-amd64.tar
  3. On the node that you want to be the master node, navigate to the extracted file.
    cd <dsp-version>
  4. If you are installing DSP with SELinux enabled, temporarily disable SELinux in your Linux OS. This disables SELinux until your Linux server is rebooted.
    setenforce 0
  5. From the extracted file directory, run one of the following commands. This may take up to 15 minutes to install. Keep this terminal window open. For more information about the types of flags available for use, see the Flags section on the Gravitational installation documentation.
    Command Notes
    ./install --flavor=ha Runs the DSP installer.
    ./install --accept-license --flavor=ha Runs the DSP installer and automatically accepts the license agreement printed upon completion.
    ./install --service-uid 1001 --flavor=ha Runs the DSP installer and specifies the service user.
    ./install --mount=data:/data/gravity/pv --state-dir=/data/gravity --flavor=ha Runs the DSP installer and changes the location where Gravity stores containers and state information. Defaults to /var/lib/gravity. This example stores data on the /data directory. Use this command if you do not have enough disk space in /var to support 24 hours of data retention.
  6. If you are installing DSP on a CentOS or RHEL Operating System, to prevent Gravity hosts from running out of inotify watches, increase fs.inotify.max_user_watches to 1000000 located in /etc/sysctl.d/99-sysctl.conf. If your Gravity hosts run out of inotify watches, Gravity throws an out of disk error.
    • On each node, open the 99-sysctl.conf file in /etc/sysctl.d/
    • Add the following line to the file:
    • Save your changes.
    • From the command-line of the master node, type the following command:
      sysctl -p /etc/sysctl.d/99-sysctl.conf

Open the required Gravity Ports and finish the install

Gravity is a toolkit that allows developers to package their Kubernetes clusters and apps as a tarball. All Data Stream Processor services are deployed on top of Kubernetes. Complete the following steps to open the requisite Gravity ports and complete the DSP installation.

  1. Open the required ports that are required by the Data Stream Processor.
  2. After the installation process has finished, the installer prints out a command that you must use to join the other nodes to the first master node. Copy the text after gravity join.
    Wed Oct  2 23:59:56 UTC Please execute the following join commands on target nodes:
    Role    Nodes   Command
    ----    -----   -------
    worker  2       gravity join <ip-address-master> --token=<token> --role=worker
  3. On each of the worker nodes, enter the following.
    ./join <ip-address-of-master> --token=<token> --role=worker

When a minimum of two nodes have joined your cluster, the install continues and the following things occur:

  • Checks that the system is running on a supported OS.
  • Checks that the system passes pre-installation checks such as meeting the minimum system requirements.
  • Checks that the system is not already running docker or other conflicting software
  • Checks that the system has the necessary running services and kernel modules
  • Installs Docker, Kubernetes, and other software dependencies like SCloud
  • Prepares Kubernetes to run the Data Stream Processor
  • Installs the Data Stream Processor
  • Checks that the Data Stream Processor is ready for use
    • The application status hook will be invoked - failures will be tagged as "degraded" in gravity status.

Configure the Data Stream Processor UI redirect URL

By default, the Data Stream Processor uses the IPv4 address of eth0 to derive several properties required by the UI to function properly. This will work in many but not all cases.

In the case that the eth0 network is not directly accessible (for example, it exists inside a private AWS VPC) or is otherwise incorrect, use the configure-ui script to manually define the IP or hostname that can be used to access DSP.

  1. From the master node, enter the following:
    DSP_HOST=<ip-address-of-master-node> ./configure-ui
  2. Then, enter the following:
  3. Navigate to the Data Stream Processor UI to verify your changes.
  4. On the login page, enter the following:
    User: dsp-admin
    Password: <the dsp-admin password generated from the installer>
  5. (Optional) If you need to retrieve the dsp-admin password, enter the following on your master node:

Check the status of your Data Stream Processor deployment

To check the status of your cluster, do the following:

  1. From a node, type the following.
    gravity status

A response showing the current health of your cluster is displayed.

Cluster status:     active
Application:        dspbeta, version 0.1.5-503873
Join token:     cb8155ed37115fe4f70cd896e4a0eea5
Periodic updates:   Not Configured
Remote support:     Not Configured
Last completed operation:
    * operation_install (5b6cfae2-ee66-4789-a7e9-9d257d99cea9)
      started:      Fri Sep 27 16:02 UTC (3 days ago)
      completed:    Fri Sep 27 16:02 UTC (3 days ago)
Cluster endpoints:
    * Authentication gateway:
    * Cluster management URL:
Cluster nodes:  musingbanach888
        * ip-172-31-24-68 / / worker
            Status: healthy

Cluster configuration options

You can view or change the default configurations of your cluster by using the following commands.

  • get-config
  • set-config
  • list-configs
  • get-secret
  • set-secret
  • list-secrets

To set a new configuration or secret:

  1. To set a new configuration, type the following from a node.
    ./set-config <PROPERTY_NAME> <VALUE> 
  2. To set a new secret, type the following from a node.
    ./set-secret <SECRET_NAME>
  3. Deploy the changes.

To view existing cluster configurations:

  1. From a node, type the following to see the list of configurations, including values.
  2. From a node, type the following to see the list of secret keys.

To view individual configurations or secrets:

  1. From a node, type the following to see an individual non-secret configuration property.
    ./get-config <PROPERTY_NAME>
  2. From a node, type the following to see an individual secret.
    ./get-secret <SECRET_NAME>
Hardware and Software Requirements
Port configuration requirements

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.0.0

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters