Upgrade a single node OEL installation of Splunk UBA

Perform the following steps to upgrade a single node OEL installation of Splunk UBA.

OEL Leapp regularly updates its base operating system to the latest version of OEL 8.x. If you are a Splunk UBA customer using OEL, refrain from upgrading OEL until you verify the OEL version Leapp will upgrade your environment to is supported by the target UBA version. For information on the latest OEL version, see https://docs.oracle.com/en/operating-systems/oracle-linux/8 /.

Prerequisites

Complete the following steps to ensure your system is correctly set up for upgrading Splunk UBA:

1. Review the new features in this version of Splunk UBA for any changes that might need to be addressed before upgrading. See What's new in the Release Notes manual.
2. Confirm you meet the Upgrade Splunk UBA prerequisites. Make sure that the prerequisites are verified on each server in the distributed deployment.
3. Make sure the correct hadoop ports are open. See, Inbound networking port requirements.
4. If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), preserve the associated certificate before upgrading.
   Use the following command to export the certificate:

   . /opt/caspida/bin/CaspidaCommonEnv.sh
   sudo keytool -exportcert -alias "splunk es" -keystore $JAVA_HOME/lib/security/cacerts -rfc -file ~/splunk-es_cacert.pem

   For more information on sending audit data to ES see, Send Splunk UBA audit events to Splunk ES in the Send and Receive Data from the Splunk Platform manual.
5. If you have enabled validation of the SSL certificate from Splunk ES, export the SSL certificate used for validating datasources from the Splunk Enterprise platform:

   . /opt/caspida/bin/CaspidaCommonEnv.sh
   sudo keytool -exportcert -alias "SplunkESRootCA" -keystore $JAVA_HOME/lib/security/cacerts -rfc -file ~/SplunkESRootCA.pem

6. If you have enabled the integration that sends UBA events to Splunk ES, add connection_host = ip to the HTTP Event Collector (HEC) inputs.conf on the ES search head.
   For example:
   

    /opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

   This ensures that the host field remains the sender's (UBA) IP address instead of the default HEC host and port.
7. Customers with existing UBA-ES integrations must comment out or remove the previously configured [tcp-ssl:10008] stanza from the Splunk_TA_ueba inputs.conf on the Splunk ES search head to avoid having an unused listener.

Stop all UBA services and archive

Switch to the caspida user and perform the following steps to archive the older version of Splunk UBA:

1. Stop all the caspida services:

   /opt/caspida/bin/Caspida stop-all

2. Archive the older version of UBA. REPLACE <old-uba-version> with your UBA version:

   sudo mkdir -p /var/vcap/release_archives
   sudo mv /opt/caspida /var/vcap/release_archives/caspida-<old-uba-version>	
   sudo mkdir -p /opt/caspida && sudo chown caspida:caspida /opt/caspida
   sudo chown caspida:caspida /var/vcap/release_archives
   sudo chown caspida:caspida /var/vcap/release_archives/caspida-<old-uba-version>

Upgrade your OEL operating system from 8.x to 8.10

An operating system upgrade is not required to upgrade to UBA version 5.4.1.

Perform the following tasks on each Splunk UBA node to upgrade your OEL operating system to version 8.10:

1. Log in to the server as the root user.
2. Perform the initial check of the system.
   1. Set system locale to en_US.UTF-8 inside /etc/locale.conf file:

      LANG="en_US.UTF-8"
      LC_CTYPE="en_US.UTF-8"

   2. Source the /etc/locale.conf file:

      source /etc/locale.conf

3. Remove the rootcerts package before upgrading to OEL 8.10:

   sudo rpm -e --nodeps rootcerts-1:20201201.00-2.mga8.noarch

4. Perform the upgrade:

   sudo yum update -y

5. Manually reboot the system

   reboot

Perform UBA upgrade

Perform the following tasks to upgrade Splunk UBA on a single node.

1. Switch to caspida user and perform the following steps to archive the older version of UBA.
1. On the management node, download the latest UBA branch build splunk-uba-software-upgrade-package_541.tgz and untar the UBA bits to /home/caspida directory:

   $ tar xvzf /home/caspida/splunk-uba-software-upgrade-package_541.tgz
   Splunk-UBA-Platform-5.4.1-20240821-19277151.tgz
   Splunk-UBA-Platform-5.4.1-20240821-19277151.tgz.md5sum
   uba-ext-pkgs-5.4.1.tgz
   uba-ext-pkgs-5.4.1.tgz.md5sum

2. Download and untar the UBA 5.4.1 Platform build inside the /opt/caspida/ folder:

   tar xvzf /home/caspida/Splunk-UBA-Platform-5.4.1-20240821-19277151.tgz -C /opt/caspida

2. (Optional) Follow the steps to turn on FIPS compliance. See Turn on FIPS compliance.




3. Run the UBA upgrade script, using the path to your archived UBA from step 2 in the Stop all UBA services and archive section.

   The path-to-prev-uba-archive might be /var/vcap/release_archives/caspida- depending on your archived UBA version number.

   /opt/caspida/upgrade/utils/upgrade_uba.sh -p /var/vcap/release_archives/caspida-<old-uba-version> -e /home/caspida/uba-ext-pkgs-5.4.1.tgz

   The command installs the new Splunk UBA software, restarts Splunk UBA, and then restarts the data sources.

If you have previously imported an output connector certificate, re-import the certificate. See, Configure the Splunk platform to receive data from the Splunk UBA output connector in the Send and Receive Data from the Splunk Platform manual.

Turn on FIPS compliance

Turning on FIPS compliance is an optional step when installing or upgrading Splunk UBA.

Federal Information Processing Standard (FIPS) compliance is available with Splunk UBA version 5.4.0 and higher. Complete the following steps to turn on FIPS on each Splunk UBA node before running the upgrade script in the Perform UBA upgrade section.

Make sure that the operating system is on the target version before turning on FIPS.

1. Run the following command to check the current status of FIPS:

   sudo fips-mode-setup --check

2. On each node, run the following command to turn on FIPS:

   sudo fips-mode-setup --enable

3. After successfully turning on FIPS, reboot the system:

   sudo reboot

4. Confirm FIPS is turned on:

   sudo fips-mode-setup --check

5. You can also verify the status using the following command.

   You see a 1 if FIPS is turned on, otherwise 0.

   cat /proc/sys/crypto/fips_enabled

Re-import the certificate used for sending UBA audit events to Splunk ES

If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), and you preserved the associated certificate in the Before you begin step, you can now re-import that certificate. Use the following command to re-import the certificate:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -import -alias "splunk es" -keystore $JAVA_HOME/lib/security/cacerts -file ~/splunk-es_cacert.pem

Re-import the SSL certificate used for validating datasources from the Splunk Enterprise platform

If you have enabled the integration that sends UBA audit data to Splunk Enterprise Security (ES), use the following command to re-import the SSL certificate:

. /opt/caspida/bin/CaspidaCommonEnv.sh
sudo keytool -import -alias "SplunkESRootCA" -keystore $JAVA_HOME/lib/security/cacerts -file ~/SplunkESRootCA.pem

For more information on SSL certificate validation, see Configure flag to enable or disable Splunk SSL certificate validation.

Apply security patches on your Linux operating system

Perform the following tasks to apply the latest Linux operating system security patches:

1. Log in to the Splunk UBA server as the caspida user.
2. Run the following command to stop Splunk UBA and all services:

   /opt/caspida/bin/Caspida stop-all

3. Run the following commands to check for any available security updates:

   sudo yum updateinfo list security all
   sudo yum updateinfo list sec
   

4. Run the following command to resolve glibc package dependencies:

   sudo yum update glibc-devel

5. Run the following command to update all packages with the available security updates:

   sudo yum update --security -y
   sudo yum --security update-minimal
   

6. Reboot the system:

   sudo reboot

7. Run the following command to start Splunk UBA and all services:

   /opt/caspida/bin/Caspida start-all

Troubleshoot the OEL upgrade

When upgrading OEL, environment specific issues can arise. See the following known issue and how to work around it.

During the sudo yum update -y step, you might encounter an error message similar to the following:

Error: Error downloading packages:
 kernel-4.18.0-553.16.1.el8_10.x86_64: Download failed: Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/kernel-4.18.0-553.16.1.el8_10.x86_64.rpm [error setting certificate verify locations:
   CAfile: /etc/pki/tls/certs/ca-bundle.crt
   CApath: none]

To resolve this, complete the following steps:

1. Identify the path to the certificate bundle. For example:

   /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

2. Run the following command to manually create the missing symbolic link:

   sudo ln -s /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /etc/pki/tls/certs/ca-bundle.crt

3. Rerun the upgrade step:

   sudo yum update -y

Next steps

You can Verify a successful upgrade of Splunk UBA.

By default, the caspida user is given ALL access in /etc/sudoers during Splunk UBA installation and upgrade. If you want to restrict sudo access for the caspida user after Splunk UBA is upgraded, see Restrict sudo access for the caspida account.