Install Splunk UBA on several Linux servers
Install Splunk UBA on several servers with Oracle Enterprise Linux (OEL), Red Hat Enterprise Linux (RHEL), or CentOS installed.
Follow these instructions to perform a bare metal installation of Splunk UBA 5.0.0 for the first time. If you already have Splunk UBA, do not follow the instructions on this page. Instead, follow the appropriate upgrade instructions to obtain your desired release. See How to install or upgrade to this release of Splunk UBA.
Prerequisites for installing Splunk UBA on several Linux servers
- You must install Splunk UBA on a server that is running a supported operating system. See Operating system requirements.
- Make sure your Red Hat Enterprise Linux license includes the proper subscription names. See Additional RHEL requirements.
- Determine the interface of your system network configuration, for example
eth0
oren0
. You will need this information later in the installation process. - The
yum-config-manager
command must be available on your system. If it is not, install theyum-utils
package by running the following command:yum install yum-utils
- The
firewalld
package must be installed on your system. Usefirewall-cmd --state
orsystemctl status firewalld
to check iffirewalld
is installed. Use the following command to installfirewalld
if you don't have it:yum install firewalld
Configure permissions for and prepare the caspida user
Enable sudo permissions for the caspida user.
- Edit the
/etc/sudoers
file. - If the following line exists, comment the line
Defaults requiretty
. - Add the following lines at the end of the
/etc/sudoers
file.Thecaspida ALL=(ALL) NOPASSWD:ALL Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin
/etc/sudoers
file is read sequentially, so placing these lines at the end ensures that there is no impact to the caspida user from any existing accounts or group permissions. - Add the caspida user to the systems. The caspida user needs to have the same UID and GID on all the systems. Pick a UID and GID that is available on all the systems. For example, assuming UID and GID 2018 is available on all nodes:
groupadd --gid 2018 caspida useradd --uid 2018 --gid 2018 -m -d /home/caspida -c "Caspida User" -s /bin/bash caspida
- Set the password for caspida user:
passwd caspida
Obtain the installation packages
Download the following Splunk UBA software and RHEL packages.
Splunk UBA 5.0.0 requires files from the Splunk UBA 5.0.4 installation package in order to complete the installation on RHEL, OEL, or CentOS 7.8 or later. Follow the installation instructions carefully and make sure you do not skip the steps to obtain and extract files from the Splunk UBA 5.0.4 installation package. At the end of the installation, you will be running Splunk UBA 5.0.0. You can then upgrade to the appropriate Splunk UBA version.
- Obtain the Splunk UBA 5.0.0 software:
- Go to the Splunk UBA Software Installation page on Splunkbase.
- Download the file to the
/home/caspida
directory. The name of the package issplunk-uba-software-installation-package_50.tgz
.
- Obtain the Splunk UBA 5.0.4 software:
- Go to the Splunk UBA Software Update page on Splunkbase.
- Select 5.0.4 from the Version history tab.
- Download the file to the
/home/caspida
directory. The name of the archive file issplunk-uba-software-upgrade-package_504.tgz
.
Use these packages for all supported Linux environments. The package can be used for RHEL, OEL, and CentOS environments.
Prepare all servers in your distributed Linux environment
Perform these steps on every server node in the distributed deployment.
- From the command line, log in to the server as the root user, or log in as a different user then use
su
orsudo
to gain root user privileges. - Find the additional 1TB disk or disks using the
fdisk -l
command. The nodes that are running Spark services should have two 1TB disks. See Disk space and memory requirements for a summary of where Spark is running per deployment.
For example, you may see two disks named/dev/sdb
and/dev/sdc
. - Partition and format the partition on each disk found in step 2.
- Partition and format the partition on the
/dev/sdb
disk using the following series of commands. Verify that thealign-check opt 1
command returns1 aligned
.parted -a optimal /dev/sdb mklabel gpt mkpart primary ext4 2048s 100% align-check opt 1 quit
- Format the partition using the
mkfs
command.mkfs -t ext4 /dev/sdb1
- On nodes that have a second disk, repeat the commands to partition and format the partition on
/dev/sdc
:parted -a optimal /dev/sdc mklabel gpt mkpart primary ext4 2048s 100% align-check opt 1 quit
- Format the partition using the
mkfs
command. When prompted, confirm that you want to continue.mkfs -t ext4 /dev/sdc1
- Partition and format the partition on the
- Get the block ID for each disk using the
blkid
command. For example, to get the block IDs for/dev/sdb1
and/dev/sdc1
in our example:An example block ID might be:blkid -o value -s UUID /dev/sdb1 blkid -o value -s UUID /dev/sdc1
5c00b211-e751-4661-91c4-60d9f9315857
. - Create new
/var/vcap
and/var/vcap2
directories. For example, on a node with a single 1TB disk:mkdir -p /var/vcap
Or on a node with two 1TB disks:
mkdir -p /var/vcap /var/vcap2
- Add the block ID for the
/var/vcap
partition to the/etc/fstab
directory. For example, on a node with a single 1TB disk:UUID=e1af8814-9b12-4c69-a947-18af370c7dd1 /var/vcap ext4 defaults 0 0
On a node with two 1TB disks:
UUID=e1af8814-9b12-4c69-a947-18af370c7dd1 /var/vcap ext4 defaults 0 0 UUID=f142f182-27c6-4002-b0bb-941fbedce17d /var/vcap2 ext4 defaults 0 0
- Mount the file systems using the
mount -a
command. - Verify that the 1TB disks are mounted correctly using the
df -h
command. For example:root# df -h Filesystem Size Used Avail Use% Mounted on ... /dev/sdc1 493G 77M 467G 1% /var/vcap2 /dev/sdb1 985G 43G 892G 5% /var/vcap ...
- Inherit the permissions for the root user. On a node with a single 1TB disk:
chmod 755 /var/vcap chown root:root /var/vcap
Or on a node with two 1TB disks:
chmod 755 /var/vcap /var/vcap2 chown root:root /var/vcap /var/vcap2
- Make a directory for caspida software packages.
This should be different from caspida home directory (
/home/caspida
).mkdir /opt/caspida chown caspida:caspida /opt/caspida chmod 755 /opt/caspida
- Set the following environment variables in either the
/etc/default/locale
or/etc/locale.conf
file:export LC_ALL="en_US.UTF-8" export LC_CTYPE="en_US.UTF-8"
- If your environment contains both internal and external IP addresses, be sure to use the internal IP address when configuring Splunk UBA. You can use the
ip route
command to help you determine this. - Verify that the host name resolves using the
nslookup <host name>
command. If it does not, verify your host name lookup and DNS settings. See Configure host name lookups and DNS. Ifnslookup
command is not available, installbind-utils
:yum install bind-utils
- Verify that the system date, time and time zone are correct using the
timedatectl
command, as shown below. The time zone in Splunk UBA should match the time zone configured in Splunk Enterprise.root# timedatectl status Local time: Mon 2019-04-08 14:30:02 UTC Universal time: Mon 2019-04-08 14:30:02 UTC RTC time: Mon 2019-04-08 14:30:01 Time zone: UTC (UTC, +0000) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a
Use the
timedatectl
command to change the time zone. For example, to change the time zone to UTC:Refer to the documentation for your specific operating system to configure NTP synchronization. Use thetimedatectl set-timezone UTC
ntpq -p
command to verify that NTP is pointing to the desired server. - Modify
/etc/sysconfig/selinux
and setSELINUX=permissive
.
With SELINUX set toenforced
, certain actions during installation and upgrade (for example, access to particular files) can be blocked. Set SELINUX topermissive
to allow Splunk UBA the necessary access so that actions are not blocked, but instead logged in the audit logs. - Verify that
/proc/sys/net/bridge/bridge-nf-call-iptables
exists on your system and the content ofbridge-nf-call-iptables
is1
. Run the following command to verify:cat /proc/sys/net/bridge/bridge-nf-call-iptables
Your situation Take this action /proc/sys/net/bridge/bridge-nf-call-iptables
exists on your system and the content is1
.- Run the following command to make sure this setting is preserved through any reboot operations:
echo net.bridge.bridge-nf-call-iptables=1 > /etc/sysctl.d/splunkuba-bridge.conf
- Go to Step 17.
/proc/sys/net/bridge/bridge-nf-call-iptables
exists on your system but the content is not1
.- Run the following commands to set the content of the
bridge-nf-call-iptables
:sysctl -w net.bridge.bridge-nf-call-iptables=1
- Run the following command to ensure that the settings persist through any reboot operations:
echo net.bridge.bridge-nf-call-iptables=1 > /etc/sysctl.d/splunkuba-bridge.conf
- Go to Step 17.
/proc/sys/net/bridge/bridge-nf-call-iptables
does not exist on your system.- Run the following commands to create the file and ensure that it is loaded on reboot:
modprobe br_netfilter echo br_netfilter > /etc/modules-load.d/br_netfilter.conf
- Run the following commands to create and set the content of the
bridge-nf-call-iptables
:sysctl -w net.bridge.bridge-nf-call-iptables=1
- Run the following command to ensure that the settings persist through any reboot operations:
echo net.bridge.bridge-nf-call-iptables=1 > /etc/sysctl.d/splunkuba-bridge.conf
- Go to Step 17.
- Run the following command to make sure this setting is preserved through any reboot operations:
- Check the permissions of
/etc/sysctl.d/splunkuba-bridge.conf
and verify that it is readable by the caspida user. For example:[caspida@ubanode1 ~]$ ls -l /etc/sysctl.d/splunkuba-bridge.conf -rw-r--r--. 1 root root 37 Aug 18 2020 /etc/sysctl.d/splunkuba-bridge.conf
- Verify that IPv6 drivers are available. To do this, check that
/proc/sys/net/ipv6/
exists. For example:root# ls -l /proc/sys/net/ipv6/ total 0 -rw-r--r-- 1 root root 0 Mar 12 16:52 anycast_src_echo_reply -rw-r--r-- 1 root root 0 Mar 12 16:52 auto_flowlabels -rw-r--r-- 1 root root 0 Mar 12 16:52 bindv6only dr-xr-xr-x 1 root root 0 Mar 12 16:52 conf -rw-r--r-- 1 root root 0 Mar 12 16:52 flowlabel_consistency -rw-r--r-- 1 root root 0 Mar 12 16:52 flowlabel_state_ranges -rw-r--r-- 1 root root 0 Mar 12 16:52 fwmark_reflect dr-xr-xr-x 1 root root 0 Mar 12 16:52 icmp -rw-r--r-- 1 root root 0 Mar 12 16:52 idgen_delay -rw-r--r-- 1 root root 0 Mar 12 16:52 idgen_retries -rw-r--r-- 1 root root 0 Mar 12 16:52 ip6frag_high_thresh -rw-r--r-- 1 root root 0 Mar 12 16:52 ip6frag_low_thresh -rw-r--r-- 1 root root 0 Mar 12 16:52 ip6frag_secret_interval -rw-r--r-- 1 root root 0 Mar 12 16:52 ip6frag_time -rw-r--r-- 1 root root 0 Mar 12 16:52 ip_nonlocal_bind -rw-r--r-- 1 root root 0 Mar 12 16:52 mld_max_msf -rw-r--r-- 1 root root 0 Mar 12 16:52 mld_qrv dr-xr-xr-x 1 root root 0 Mar 12 16:52 neigh dr-xr-xr-x 1 root root 0 Mar 12 16:52 route -rw-r--r-- 1 root root 0 Mar 12 16:52 xfrm6_gc_thresh
If the IPv6 drivers exist, skip to the next step.
If IPv6 drivers do not exist on your system, check if/etc/default/grub
containsipv6.disable=1
. IPv6 drivers will not be available on a system ifipv6.disable=1
exists in/etc/default/grub
. Ifipv6.disable=1
is not present in/etc/default/grub
and IPv6 drivers do not exist, consult with your system or network administrators. You will not be able to continue with the installation.
If/etc/default/grub
containsipv6.disable=1
, perform the following tasks as root:- Remove
ipv6.disable=1
from/etc/default/grub
. - Recreate the grub config:
find /boot -name grub.cfg -exec grub2-mkconfig -o '{}' \;
- Reboot the machines. After the system comes up, make sure
/proc/sys/net/ipv6
exists.
To disable IPv6 functionality for security, networking or performance reasons, create the
/etc/sysctl.d/splunkuba-ipv6.conf
file as root. This file should contain the following content:net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1
This procedure keeps the IPv6 drivers but disables the IPv6 addressing. - Remove
- Create the
/etc/security/limits.d/caspida.conf
file and add the following security limits for thecaspida
user to this file:caspida soft nproc unlimited caspida soft nofile 32768 caspida hard nofile 32768 caspida soft core unlimited caspida soft stack unlimited caspida soft memlock unlimited caspida hard memlock unlimited
Make sure the root account does not have any security limits.
- If you are not using IPv6 on your network, edit the
/etc/yum.conf
file and add the following entry so that only IPv4 addresses are used by yum/rpm:ip_resolve=4
- Update to the latest kernel. See Operating system requirements for the specific kernel versions.
Operating System Update Instructions OEL 7.7 Perform all of the following tasks on each Splunk UBA node: - Run the following command to install
yum-utils
:sudo yum install yum-utils -y
- Run the following commands to skip certain packages if they are not available:
sudo yum-config-manager --disable pgdg94 sudo yum-config-manager --disable nodesource sudo yum-config-manager --disable rhel-7-server-rt-beta-rpms
- Obtain http://yum.oracle.com/public-yum-ol7.repo and download it to
/home/caspida
.sudo wget http://yum.oracle.com/public-yum-ol7.repo /home/caspida
- Copy
public-yum-ol7.repo
to/etc/yum.repos.d
:sudo cp /home/caspida/public-yum-ol7.repo /etc/yum.repos.d/public-yum-ol7.repo
- Enable the repositories:
sudo yum-config-manager --enable ol7_UEKR5 sudo yum-config-manager --enable ol7_addons
- Update the kernel to the latest versions.
- Run the following command if you are using OEL 7.7:
sudo yum update --releasever=7.7 --exclude="zookeeper redis-server redis-tools influxdb nodejs nodejs-docs postgres*" -y
- Run the following command if you are using OEL 7.8 or 7.9:
sudo yum update --releasever=7.8 --exclude="zookeeper redis-server redis-tools influxdb nodejs nodejs-docs postgres*" -y
- Run the following command if you are using OEL 7.7:
RHEL 7.7, 7.8, or 7.9 Perform all of the following tasks on each Splunk UBA node: - Enable the repos for required packages:
- Run the following commands if you are using RHEL 7.7:
sudo subscription-manager repos --enable=rhel-7-server-extras-rpms sudo subscription-manager repos --enable=rhel-7-server-eus-rpms sudo subscription-manager repos --enable=rhel-7-server-rpms sudo subscription-manager repos --enable=rhel-7-server-optional-rpms
- Run the following commands if you are using RHEL 7.8 or 7.9:
sudo subscription-manager repos --enable=rhel-7-server-extras-rpms sudo subscription-manager repos --enable=rhel-7-server-rpms sudo subscription-manager repos --enable=rhel-7-server-optional-rpms
- Run the following commands if you are using RHEL 7.7:
- Run the following command to install
yum-utils
:sudo yum install yum-utils -y
- Run the following commands to skip certain packages if they are not available:
sudo yum-config-manager --disable pgdg94 sudo yum-config-manager --disable nodesource sudo yum-config-manager --disable rhel-7-server-rt-beta-rpms
- Update the kernel to the latest versions.
- Run the following command if you are using RHEL 7.7:
sudo yum update --releasever=7.7 --exclude="zookeeper redis-server redis-tools influxdb nodejs nodejs-docs postgres*" -y
- Run the following command if you are using RHEL 7.8 or 7.9:
sudo yum update --releasever=7.8 --exclude="zookeeper redis-server redis-tools influxdb nodejs nodejs-docs postgres*" -y
- Run the following command if you are using RHEL 7.7:
CentOS
(latest kernel)Run the following command to update your CentOS kernel to the latest available version:
sudo yum update -y
- Run the following command to install
- If you have any firewall configuration enabled, disable the configuration and verify that port 9002 is open. Run the following command:
You can re-enable your firewall settings after the setup is complete.
systemctl disable firewalld
- Restart the system.
init 6
- After the system restarts, use the following command to verify that the host name matches your host name lookup and DNS settings. See Configure host name lookups and DNS.
hostname --fqdn
Install Splunk UBA on each Linux server
Perform these steps on every server node in the distributed deployment.
- Log in to the command line as the caspida user using SSH.
- Verify that the caspida user has umask permissions set to 0022 or 0002.
umask
If the returned values are not supported, edit the~/.bash_profile
and the~/.bashrc
files and appendumask 0022
- Copy the file for Splunk UBA Software Installation from Splunkbase to the
/home/caspida
directory. - Untar the file for Splunk UBA Software Installation in
/home/caspida
directory.tar xvzf /home/caspida/splunk-uba-software-installation-package_50.tgz
- Run the following command to untar the Splunk UBA platform software to the
/opt/caspida
directory.tar xvzf /home/caspida/splunk-uba-software-installation-package_50/Splunk-UBA-Platform-5.0.0-20191015-000100.tgz -C /opt/caspida/
- Follow the instructions in the table for your operating system to untar the Splunk UBA packages to the
/home/caspida
directory.Your operating system Instructions CentOS Run the following command to check the operating system version:
cat /etc/centos-release
- If your CentOS version is earlier than 7.8, run the following command to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/
- If your CentOS version is 7.8 or later, run the following commands to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/ tar xzvf /home/caspida/splunk-uba-software-update_504.tgz tar xvzf /home/caspida/Splunk-UBA-5.0-Overlay-Packages-RHEL-7.8.tgz -C /home/caspida/Splunk-UBA-5.0-Packages-RHEL-7.7
OEL Run the following command to check the operating system version:
cat /etc/oracle-release
- If your OEL version is earlier than 7.8, run the following command to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/
- If your OEL version is 7.8 or later, run the following commands to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/ tar xzvf /home/caspida/splunk-uba-software-update_504.tgz tar xvzf /home/caspida/Splunk-UBA-5.0-Overlay-Packages-RHEL-7.8.tgz -C /home/caspida/Splunk-UBA-5.0-Packages-RHEL-7.7
RHEL Run the following command to check the operating system version:
cat /etc/redhat-release
-
If your RHEL version is earlier than 7.8, run the following command to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/
- If your RHEL version is 7.8 or later, run the following commands to untar the Splunk UBA packages.
tar xvzf /home/caspida/splunk-uba-rhel-7x-software-for-bare-metal-installation_50/Splunk-UBA-5.0-Packages-RHEL-7.7.tgz -C /home/caspida/ tar xzvf /home/caspida/splunk-uba-software-update_504.tgz tar xzvf /home/caspida/splunk-uba-software-update_504/Splunk-UBA-5.0-Overlay-Packages-RHEL-7.8.tgz -C /home/caspida/Splunk-UBA-5.0-Packages-RHEL-7.7
The overlay package contains missing dependencies required to complete the installation for Splunk UBA 5.0.0. When you complete the installation process, you will be running Splunk UBA 5.0.0. You can then upgrade to the desired 5.0.x version.
- If your CentOS version is earlier than 7.8, run the following command to untar the Splunk UBA packages.
- Run the installation script.
The log file is
/opt/caspida/bin/installer/redhat/INSTALL.sh /home/caspida/Splunk-UBA-5.0-Packages-RHEL-7.7
/var/log/caspida/install.log
. - If the network interface is not the default
eth0
, edit configuration file/etc/caspida/local/conf/uba-site.properties
and add the following entry with the corresponding interface:system.network.interface=<interface>
- Generate SSH keys using the
ssh-keygen -t rsa
command. Press enter for all the prompts and accept all default values. For example:[caspida@ubahost-001]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/caspida/.ssh/id_rsa): Created directory '/home/caspida/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/caspida/.ssh/id_rsa. Your public key has been saved in /home/caspida/.ssh/id_rsa.pub. The key fingerprint is: SHA256:Ohe1oSpUtNT8siJzvn2lFLrHmVH7JGKke+c/5NRFb/g caspida@ubahost-001
- Add the SSH keys to the server and adjust the permissions to allow the server to access them.
cat /home/caspida/.ssh/id_rsa.pub >> /home/caspida/.ssh/authorized_keys chmod 600 /home/caspida/.ssh/authorized_keys
- Copy the SSH keys from every server from
/home/caspida/.ssh/id_rsa.pub
into the/home/caspida/.ssh/authorized_keys
file on every server in the distributed deployment. After this is complete, eachauthorized_keys
file on each server should have all the SSH keys for every server in the deployment listed. - Test to verify that the SSH connections do not require a password. Connect to each server without a password with the host name or internal IP using SSH to create trusted connections between the servers. After you confirm the connection does not require a password, use
exit
to terminate the SSH connection. You must complete this step before continuing with setup.ssh <node1>; exit ssh <node2>; exit ssh <node3>; exit
- When prompted, confirm that you want to continue. For example, the sample output will look similar to the following.
[caspida@ubahost-001]$ ssh uba1 The authenticity of host 'uba1 (172.31.35.204)' can't be established. ECDSA key fingerprint is af:12:54:60:f5:36:c2:36:9d:56:b2:52:9f:cb:73:bc. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'uba1,172.31.35.204' (ECDSA) to the list of known hosts.
Complete the distributed Linux Splunk UBA installation on the management server
After completing the previous steps on all server nodes in the distributed deployment, perform the following steps on the management server. For example, uba-node01
.
- Check the system status with the
uba_pre_check.sh
shell script. You must specify the host name of each Splunk UBA node in the command, separated by spaces. For example, run the following command in a 3-node deployment and be sure to replace<node1> <node2> <node3>
with the actual host names of your Splunk UBA nodes.See Check system status before and after installation for more information about the script./opt/caspida/bin/utils/uba_pre_check.sh <node1> <node2> <node3>
- Run the setup script.
/opt/caspida/bin/Caspida setup
- When prompted, accept the license agreement and confirm removal of existing metadata.
- When prompted, type a comma-separated list of host names for your distributed installation. For example, specify the following in a 3-node deployment and be sure to replace
<node1> <node2> <node3>
with the actual host names of your Splunk UBA nodes:<node1>,<node2>,<node3>
- When prompted, confirm that you want to continue setting up Splunk UBA.
- The log file is
/var/log/caspida/caspida.out
- After setup completes:
- Open a web browser and log in to the public IP address with the default admin credentials to confirm a successful installation. The default username is
admin
and password ischangeme
. See Secure the default account after installing Splunk UBA for information about the default accounts provided with Splunk UBA and how to secure them. - See Verify successful installation for more information about verifying a successful installation.
- Open a web browser and log in to the public IP address with the default admin credentials to confirm a successful installation. The default username is
Install Splunk UBA on several Amazon Web Services instances | Verify successful installation |
This documentation applies to the following versions of Splunk® User Behavior Analytics: 5.0.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.4.1
Feedback submitted, thanks!