Upgrade a distributed AMI or OVA installation of Splunk UBA

Before upgrading Splunk UBA in a distributed deployment, confirm you meet the Upgrade Splunk UBA prerequisites. Make sure that the prerequisites are verified on each server in the distributed deployment.

Perform UBA upgrade

​​Install Splunk UBA 5.1.0.1 on the management node only. The upgrade script will update all relevant files on the other Splunk UBA nodes. Ensure that Splunk UBA is running before you upgrade.

To obtain and install Splunk UBA 5.1.0.1, perform the following tasks on the management node:

1. Obtain the Splunk UBA Software Update and download the file to the home/caspida directory. Select version 5.1.0.1 from the drop-down list. The downloadable archive file is named splunk-uba-software-upgrade-package_5101.tgz.
2. Extract the archive with the following command:

   tar xfz /home/caspida/splunk-uba-software-upgrade-package_5101.tgz -C /home/caspida
   

   The following files are extracted:
   • splunk-uba-software-update-000009-5101.tgz
   • splunk-uba-software-update-000009-5101.tgz.md5sum
3. Extract the software update package in the /home/caspida directory:

   tar xfz /home/caspida/splunk-uba-software-update-000009-5101.tgz -C /home/caspida

4. Run the following command:

   sed -i 's|\\$? -ne 0|\\$? -e 100|g' /home/caspida/patch_uba_5101/bin/utils/patch_uba.sh

5. Apply the patch with the following command:

   /home/caspida/patch_uba_5101/bin/utils/patch_uba.sh -p /home/caspida/patch_uba_5101

   The command installs the new Splunk UBA software, restarts Splunk UBA, and then restarts the data sources.

Apply security patches on Ubuntu

The Splunk UBA AMI and OVA images use Ubuntu as the operating system. Perform the following tasks to apply the latest security patches to your Ubuntu operating system:

Applying the security patches can take up to one hour.

1. Log in to the Splunk UBA management node as the caspida user.
2. On the management node, run the following command to stop Splunk UBA and all services:

   /opt/caspida/bin/Caspida stop-all

3. On each Splunk UBA node, perform the following tasks:
   1. Log in as the caspida user.
   2. Run the following commands to install latest unattended-upgrades package:

      sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 6A030B21BA07F4FB
      sudo apt update
      sudo apt install unattended-upgrades
      

      If you see the following prompt, select keep the local version currently installed:

      What do you want to do about modified configuration file 50unattended-upgrades?

   3. Run the following command:

      sudo apt autoremove

   4. Edit the /etc/apt/apt.conf.d/50unattended-upgrades file and un-comment the following line:

      Skip this step if you have previously applied security patches to your Ubuntu environment following these instructions.

      "${distro_id}:${distro_codename}-security";

      Leave all other lines commented out.
   5. Run the following command:

      sudo unattended-upgrade -d

   6. Edit the /etc/init.d/zookeeper-server file and change su to runuser in all of the following lines:

      Skip this step if you have previously applied security patches to your Ubuntu environment following these instructions.

      Before:

      su -s /bin/bash zookeeper -c "${DAEMON_SCRIPT} start"
      su -s /bin/bash zookeeper -c "${DAEMON_SCRIPT} stop"
      su -s /bin/bash zookeeper -c "zookeeper-server-initialize $*"
      

      After:

      runuser -s /bin/bash zookeeper -c "${DAEMON_SCRIPT} start"
      runuser -s /bin/bash zookeeper -c "${DAEMON_SCRIPT} stop"
      runuser -s /bin/bash zookeeper -c "zookeeper-server-initialize $*"
      

   7. Reboot the system:

      sudo reboot

4. On the management node, run the following command to start Splunk UBA and all services:

   /opt/caspida/bin/Caspida start-all

Troubleshoot the Ubuntu upgrade

While upgrading Ubuntu some environment specific issues can arise. See the following known issues and how to work around them.

Snapd

You might encounter the following error message if snapd is in an unexpected state:

FileNotFoundError: [Errno 2] No such file or directory: 'snap'

Workaround steps:

1. Remove and purge the snapd package:
   sudo apt purge snapd
2. Reinstall the snapd package:
   sudo apt install snapd
3. Re-run the upgrade_ubuntu.sh script.

Zookeeper server missing

You might encounter the following error if zookeeper is upgraded to a newer version, causing the removal of the zookeeper-server: Can't select installed nor candidate version from package 'zookeeper-server' as it has neither of them

Workaround steps:
Skip holding this package.

1. Open and edit /opt/caspida/upgrade/utils/upgrade_ubuntu.sh using a text editor.
2. Modify line 177 from the following:
   CMD="apt-mark hold zookeeper zookeeper-server redis-server redis-tools"
   To appear as follows:
   CMD="apt-mark hold redis-server redis-tools"
3. As applicable, repeat for all nodes in your cluster.
4. Re-run the upgrade_ubuntu.sh script.

Next steps

Verify a successful upgrade of Splunk UBA.