Restore Splunk UBA using the restore script

After you have created a backup, restore Splunk UBA using the /opt/caspida/bin/utils/uba-restore.sh script. View the command line options by using the --help option.

See the following table for the options you can use in the script:

Option	Description
--dateformat %FORMAT%	Override the default date/time format for the backup folder name. If this option is not used, the folder name is based on ISO 8601 format YYYY-MM-DD. To specify a backup folder name in the typical format used in the United States, specify MM-DD-YYYY. Using this option also overrides the date/time format of the logging messages.
--folder %FOLDER%	Override the source folder to perform the restore. By default the script looks for the restore archive in the default /var/vcap/ubabackup directory. If you used the --folder option when creating the backup to store the backup in a different directory, specify that same directory using the --folder option when restoring Splunk UBA.
--log-time	Add additional logging for how long each section takes, including all function calls and tasks. Use this option to help troubleshoot issues if your restore is taking a long time.
--no-data	Only restore the configuration. Use this option when restoring a backup that was created using --no-data flag.
--no-time	Don't log the amount of time taken for each task.
--reset-admin-pass	Reset the admin password to the system default changeme.
--skip-start-live-ds	Don't start the live data sources.
--validate	Validate that a restore can be successfully performed, but do not actually perform the restore.

Example restore workflow

The following is an example of a restore workflow:

1. Log in to the management node of your Splunk UBA deployment as caspida using SSH.
2. Navigate to the /opt/caspida/bin/utils folder:

   cd /opt/caspida/bin/utils

3. Run the restore script. This example is restoring from a backup file in the /home/caspida directory from a single node system to another single node system. The following shows the command and its output. Some repetitive portions of the command output are truncated for brevity:

   caspida@ip-172-31-29-247:~/bin$ ./uba-restore.sh --folder /var/vcap/ubabackup/2020-10-08_07-44-59
   UBA Restore Script - Version 2.1.5
   Restore started at: Thu Oct  8 15:25:07 UTC 2020
   Restore running on: uba-restore.splunk.com
   Logfile:            /var/log/caspida/uba-restore-2020-10-08_15-25-07.log
   Script Name:        uba-restore.sh
   Script SHA:         fa544f7bfc291dcc83b3455072682a77167b3de6f1a014b07fff042674c47c4a (sha256sum)
   Parsing any CLI args
   - Set source folder to /var/vcap/ubabackup/2020-10-08_07-44-59
   UBA version: 5.0.3
   Node Count: 1
   Backup Node Count: 1
   Validating summary file
   WARNING: The hostnames from the backup/restore hosts differ, this will be a migration
   > Time taken: 0.111 seconds
   Validating backup files
   > Time taken: 0.012 seconds
   Validating UBA version
   > Time taken: 0.022 seconds
   Execution Mode: Migration
   Will restore/migrate from a 1-node backup to a 1-node cluster
   Checking hypervisor and network configuration
   > Time taken: 0.018 seconds
   Testing SSH connectivity to UBA nodes
   > Time taken: 1.327 seconds
   Retrieving system information for each UBA node
   > Time taken: 5.272 seconds
   Attempting to retrieve the IP of each node (new)
   > Time taken: 0.009 seconds
   Attempting to retrieve the IP of each node (old)
   > Time taken: 0.011 seconds
   Stopping UBA (stop-all)
   > Time taken: 223.002 seconds
   Starting PostgreSQL
   > Time taken: 3.143 seconds
   Logging PostgreSQL sparkserverregistry table (pre-restore)
   > Time taken: 0.392 seconds
   Restoring PostgreSQL caspidadb database on UBA node 1 (ip-172-31-29-247)
   > Time taken: 12.929 seconds
   Restoring PostgreSQL metastore database on UBA node 1 (ip-172-31-29-247)
   > Time taken: 2.405 seconds
   Performing reset of connector stats/JMS schemas
   > Time taken: 0.772 seconds
   Stopping PostgreSQL
   > Time taken: 0.922 seconds
   Restoring timeseries data
   > Time taken: 0.644 seconds
   Backing up existing uba-system-env.sh/uba-tuning.properties
   Restoring local configurations
   > Time taken: 0.003 seconds
   Restoring UBA rules
   > Time taken: 0.007 seconds
   Restoring uba-system-env.sh/uba-tuning.properties
   Starting UBA (start-all --no-caspida)
   Checking that HDFS isnt in safe-mode
   - Safe mode is disabled
   > Time taken: 1.910 seconds
   Checking if the /user folder exists in HDFS
   - Folder exists, will attempt to remove
   Removing existing Hadoop HDFS content (attempt 1)
   > Time taken: 2.104 seconds
   Waiting for 10 seconds before proceeding
   Checking if the /user folder exists in HDFS
   - Folder has been removed, will continue
   Restoring Hadoop HDFS (this may take a while)
     - Checking status of PID 109467 (2020-10-08_15-31-41)
       - Restore is still running, please wait
       - Folder size: 178.5 K (target: 13.3 G)
     - Checking status of PID 109467 (2020-10-08_15-32-03)
       - Restore is still running, please wait
       - Folder size: 707.8 K (target: 13.3 G)
   ...
     - Checking status of PID 109467 (2020-10-08_16-23-19)
       - Restore is still running, please wait
       - Folder size: 11.5 G (target: 13.3 G)
     - Checking status of PID 109467 (2020-10-08_16-23-41)
     - Backup job has finished
   > Time taken: 3139.875 seconds
   Changing ownership of Hadoop HDFS files
   > Time taken: 30.450 seconds
   Updating location of hive warehouse
   > Time taken: 8.640 seconds
   Updating location of caspida analytics database
   > Time taken: 9.058 seconds
   Flushing existing Redis data
   > Time taken: 10.041 seconds
   Stopping Redis for Redis restore
   > Time taken: 2.352 seconds
   Restoring Redis database (parallel mode)
   Triggering Redis restore of node 1 to 1
     - Performing restore of data from UBA node 1 to UBA node 1 (ip-192-168-10-10)
     - Performing rsync of database from UBA node 1 to UBA node 1
     - Waiting for pid 87260 to finish
       - Process finished successfully
   > Time taken: 11.011 seconds
   Starting Redis after Redis restore
   > Time taken: 3.300 seconds
   Waiting for 60 seconds to let Redis settle
   Retrieving Redis information (pre-fixup) (attempt 1)
   Successfully retrieved Redis information (pre-fixup)
   > Time taken: 0.097 seconds
   Performing Redis restore fixup (attempt 1)
   > Time taken: 0.109 seconds
   Performing Redis restore rebalance (attempt 1)
   > Time taken: 0.095 seconds
   Successfully finished Redis fixup/rebalance
   Retrieving Redis information (post-fixup)
   > Time taken: 0.095 seconds
   Determining number of Redis keys (post-restore)
   - Retrieving keys from ubanode1 (ip-192-168-10-10)
   > Time taken: 0.593 seconds
   Not checking Redis key counts (old backup)
   Tuning configuration
   > Time taken: 0.611 seconds
   Stopping UBA (stop)
   > Time taken: 17.020 seconds
   Syncing UBA cluster
   > Time taken: 0.464 seconds
   Copying /opt/caspida/conf to hdfs /user/caspida/config/etc/caspida/conf
   > Time taken: 8.931 seconds
   Configuring containerization
   > Time taken: 93.969 seconds
   Starting UBA
   > Time taken: 188.828 seconds
   Testing Impala
   > Time taken: 11.236 seconds
   Logging PostgreSQL sparkserverregistry table (post-restore)
   > Time taken: 0.461 seconds
   Comparing PostgreSQL backup/restore counts/stats
   > Time taken: 3.391 seconds
   Migrating datasources
     - Migrating datasource: ASSETS
     - Migrating datasource: AWSCLOUDTRAIL
     - Migrating datasource: OSXPROCESS
     - Migrating datasource: DHCP
     - Migrating datasource: DNS
     - Migrating datasource: ESNOTABLE
     - Migrating datasource: FIREWALL
     - Migrating datasource: IDENTITIES
     - Migrating datasource: IDSIPS
     - Migrating datasource: IPFLOW
     - Migrating datasource: LINUXAUTH
     - Migrating datasource: SQUID
     - Migrating datasource: WINEVENTLOG
     - Deleting file-based datasource: file-uba-assets.csv
     - Deleting file-based datasource: file-uba-identities.csv
   > Time taken: 26.924 seconds
   Restore completed successfully
   Time taken: 1 hour(s), 6 minute(s), 41 second(s)
   

   You can review the log file in /var/log/caspida/uba-restore-<timestamp>.log.
4. If you are integrating Splunk UBA with Splunk Enterprise Security (ES), install the Splunk ES SSL certificates in the restored deployment. See Configure the Splunk platform to receive data from the Splunk UBA output connector in the Send and Receive Data from the Splunk Platform manual.