Upgrade a single node RHEL installation of Splunk UBA

Perform the following steps to upgrade a single node RHEL installation of Splunk UBA.

Before you begin

Before upgrading Splunk UBA on a single server, confirm you meet the Upgrade Splunk UBA prerequisites

If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), preserve the associated certificate before upgrading. For more info on sending audit data to ES see, Send Splunk UBA audit events to Splunk ES. Use the following command to export the certificate:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -exportcert -alias "splunk es" -keystore $JAVA_HOME/lib/security/cacerts -rfc -file ~/splunk-es_cacert.pem

If you have enabled validation of the SSL certificate from Splunk ES, export the SSL certificate used for validating datasources from the Splunk Enterprise platform:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -exportcert -alias "SplunkESRootCA" -keystore $JAVA_HOME/lib/security/cacerts -rfc -file ~/SplunkESRootCA.pem

Download the Splunk UBA upgrade software

Perform the following steps to find and download the Splunk UBA upgrade software:

1. Obtain the Splunk UBA Software Update and download the file to the /home/caspida directory. Select version 5.1.0 from the drop-down list. The downloadable archive file is named tar xfz /home/caspida/splunk-uba-software-upgrade-package_510.tgz -C /home/caspida.
2. Extract the archive with the following command:

   tar xfz /home/caspida/splunk-uba-software-upgrade-package_510.tgz -C /home/caspida

   The following files are extracted:

   • Splunk-UBA-Platform-5.1.0-20220721-000019.tgz
   • Splunk-UBA-Platform-5.1.0-20220721-000019.tgz.md5sum
   • uba-ext-pkgs-5.1.0.tgz
   • uba-ext-pkgs-5.1.0.tgz.md5sum

Stop all UBA services and archive

Switch to caspida user and perform the following steps to archive the older version of UBA.

1. On the management node, stop all the caspida services:

   /opt/caspida/bin/Caspida stop-all

2. On each node, archive the older version of UBA. REPLACE <old-uba-version> with your UBA version:

   sudo mkdir -p /var/vcap/release_archives
   sudo mv /opt/caspida /var/vcap/release_archives/caspida-<old-uba-version>
   sudo mkdir -p /opt/caspida && sudo chown caspida:caspida /opt/caspida
   sudo chown caspida:caspida /var/vcap/release_archives
   sudo chown caspida:caspida /var/vcap/release_archives/caspida-<old-uba-version>

Upgrade from RHEL 7.8 to RHEL 7.9

On each node, perform the following steps to upgrade your RHEL operating system from version 7.8 to version 7.9. If you are already using RHEL 7.9, go to Upgrade from RHEL 7.9 to 8.4.

1. Log in to the Splunk UBA management node as the caspida user.
2. In the /usr/lib/jvm/java-1.8.0/jre/lib/security/java.security file, comment out the following code:

   # jdk.tls.disabledAlgorithms=MD5, SSLv3, DSA, RSA keySize < 2048
   # jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
   # DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
   # include jdk.disabled.namedCurves
   

3. Run the following command:

   sudo yum update -y

Upgrade from RHEL 7.9 to RHEL 8.4

Perform the following steps to upgrade your RHEL operating system from version 7.9 to version 8.4.

1. On each node, perform the OS upgrade steps as the root user.
2. Perform the initial checks of the system:
1. Check that the machine is properly subscribed to the RHEL repositories:

   subscription-manager list --installed

2. Run the following commands to make sure the required repositories are enabled during the upgrade:

   subscription-manager repos --enable rhel-7-server-rpms
   subscription-manager repos --enable rhel-7-server-extras-rpms
   

3. Run the following command to set the Red Hat Subscription Manager to consume the latest RHEL 7 content:

   subscription-manager release --unset

4. Set system locale to en_US.UTF-8 in the /etc/locale.conf file:

   LANG="en_US.UTF-8"
   LC_ALL="en_US.UTF-8"
   LC_CTYPE="en_US.UTF-8"
   

5. Run the following command to source the /etc/locale.conf file:

   source /etc/locale.conf

6. Run the following command to clear any version locks:

   yum versionlock clear

3. Run the following command to install the Leapp utility:

   yum install leapp leapp-repository

4. Download the the RedHat auxiliary files from RedHat for the upgrade. This is necessary for a successful OS upgrade.
1. Go to this RedHat knowledge base article: https://access.redhat.com/articles/3664871
2. Login using your RedHat credentials.
3. Scroll to the section Community Linux distributions and download the leapp data file. At the time of this writing, the latest version was leapp-data17.tar.gz.
5. SCP the following to each node in the UBA cluster:

   scp leapp-data17.tar.gz caspida@<uba-nodeX>:/home/caspida/

6. On the node, untar the leapp-data17.tar.gz package into the /etc/leapp/files directory.

   tar -xzf leapp-data17.tar.gz -C /etc/leapp/files && rm leapp-data17.tar.gz

7. On your RHEL 7 system, perform the pre-upgrade phase. Set the target flag to the desired RedHat version. 8.4:

   leapp preupgrade --target 8.4

8. After the preupgrade step, answerfile will be generated. Provide answers before proceeding with the upgrade. Use one of the following methods to answer the questions required by Leapp:
   • Give confirmation of the sections of the leapp answerfile by editing the /var/log/leapp/answerfile file and adding True after the statement confirm = for all the respective sections.
   • Use the leapp answer command to confirm all the sections individually using section names:

     leapp answer --section question_section.confirm=answer

     For example, to confirm a True response to the question Disable pam_pkcs11 module in PAM configuration?, execute the following command:

     leapp answer --section remove_pam_krb5_module_check.confirm=True

9. Examine the report in the /var/log/leapp/leapp-report.txtfile, and manually resolve all the reported problems before proceeding with the in-place upgrade.
10. During Leapp preupgrade you may encounter following inhibitors:

    Inhibitor	Solution
    Detected loaded kernel drivers which have been removed in RHEL 8. Upgrade cannot proceed.	Check the Leapp report for detailed instructions, to know which kernel driver is causing the error. Run the following command: rmmod pata_acpi
    Possible problems with remote login using root account	Uncomment line "PermitRootLogin" in /etc/ssh/sshd_config
    Newest installed kernel not in use.	Restart your machine.
    Missing required answers in the answer file	Provide answers to each section in the answerfile. See step 8.

    All other inhibitors can be safely ignored.

11. After solving all the inhibitors and giving answers to all the sections in the /var/log/leapp/answerfile file, repeat step 7 to verify that your system is ready for upgrade:
12. On your RHEL 7 system, start the upgrade process:

    leapp upgrade --target 8.4

    1. If the system is upgradeable, Leapp downloads the necessary data and prepares an RPM transaction for the upgrade.
    2. If the system does not meet the requirements for a reliable upgrade, Leapp terminates theupgrade process and provides a record describing the issue and a recommended solution in the /var/log/leapp/leapp-report.txt file.
13. Manually reboot the system:

    reboot

    1. In this phase, the system boots into a RHEL 8-based initial RAM disk image, initramfs. Leapp upgrades all packages and automatically reboots to the RHEL 8 system.
    2. Alternatively, you can run the leapp upgrade command with the --reboot option and skip this manual step.
14. Reboot the system:

    reboot

System OS upgrade complete. The system is now at OS version 8.4.

(Optional) Upgrade from RHEL 8.4 to RHEL 8.5

1. On each node, perform the OS upgrade steps as root user:

   subscription-manager release --set=8.5
   yum update
   reboot

2. Verify the OS version is upgraded to the desired version:

   cat /etc/os-release

System OS upgrade complete. The system is now at OS version 8.5.

Perform UBA upgrade

Ensure that Splunk UBA is running before you upgrade, then perform the following tasks to upgrade Splunk UBA on a single server.

1. On the management node, download the latest UBA branch build splunk-uba-software-upgrade-package_510.tgz and untar the UBA bits to the /home/caspida directory:

   $ tar xvzf /home/caspida/splunk-uba-software-upgrade-package_510.tgz
   Splunk-UBA-Platform-5.1.0-20220721-000019.tgz
   Splunk-UBA-Platform-5.1.0-20220721-000019.tgz.md5sum
   uba-ext-pkgs-5.1.0.tgz
   uba-ext-pkgs-5.1.0.tgz.md5sum

2. Untar the UBA 5.1.0 Platform build inside the /opt/caspida/ folder:

   tar xvzf /home/caspida/Splunk-UBA-Platform-5.1.0-20220721-000019.tgz -C /opt/caspida

3. Update the list of allowed serialization classes:

   sed -i '${s/$/,com.caspida.clients.jobmanager.JobDataParams$SourceTypes,com.caspida.clients.jobmanager.JobDataParams$SourceFiles/}' /opt/caspida/conf/uba-default.properties

4. Run the UBA upgrade script, using the path to your archived UBA from the Stop all UBA services and archive step:

   The path-to-prev-uba-archive might be /var/vcap/release_archives/caspida- depending on if your archived UBA version is 5.0.5 or 5.0.5.1.

   /opt/caspida/upgrade/utils/upgrade_5.0_to_5.1.sh -p <path-to-prev-uba-archive> -e /home/caspida/uba-ext-pkgs-5.1.0.tgz

   The command installs the new Splunk UBA software, restarts Splunk UBA, and then restarts the data sources.
5. Perform these steps to finalize the hadoop upgrade:
1. On the UBA management node run the following commands:

   sudo -u hdfs hdfs dfsadmin -finalizeUpgrade

   sudo service hadoop-hdfs-namenode restart

2. Restart UBA:

   /opt/caspida/bin/Caspida stop-all

   /opt/caspida/bin/Caspida start-all

If you have previously imported an output connector certificate, re-import the certificate. See, Configure the Splunk platform to receive data from the Splunk UBA output connector in the Send and Receive Data from the Splunk Platform manual.

Re-import the certificate used for sending UBA audit events to Splunk ES

If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), and you preserved the associated certificate in the Before you begin step, you can now re-import that certificate. Use the following command to re-import the certificate:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -import -alias "splunk es" -keystore $JAVA_HOME/lib/security/cacerts -file ~/splunk-es_cacert.pem

Re-import the SSL certificate used for validating datasources from the Splunk Enterprise platform

If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), use the following command to re-import the SSL certificate:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -import -alias "SplunkESRootCA" -keystore $JAVA_HOME/lib/security/cacerts -file ~/SplunkESRootCA.pem

For more information on SSL certificate validation, see Configure flag to enable or disable Splunk SSL certificate validation.

Apply security patches on your Linux operating system

Perform the following tasks to apply the latest Linux operating system security patches:

1. Log in to the Splunk UBA server as the caspida user.
2. Run the following command to stop Splunk UBA and all services:

   /opt/caspida/bin/Caspida stop-all

3. Run the following commands to check for any available security updates:

   sudo yum updateinfo list security all
   sudo yum updateinfo list sec
   

4. Run the following command to update all packages with the available security updates:

   sudo yum update --security -y
   sudo yum --security update-minimal
   

5. Reboot the system:

   sudo reboot

6. Run the following command to start Splunk UBA and all services:

   /opt/caspida/bin/Caspida start-all

Next steps

Verify a successful upgrade of Splunk UBA.