Known issues in Splunk UBA

This version of Splunk UBA has the following known issues and workarounds.

If no issues are listed, none have been reported.

Date filed	Issue number	Description
2023-06-08	UBA-17446	Upon applying the Ubuntu security patches, postgresql got removed causing UBA unable to start Workaround: Stop all UBA Services : /opt/caspida/bin/Caspida stop-all Re-install postgres package, replace <uba ext packages> with your package folder in below command. For example, for 5.0.5 its uba-ext-pkgs-5.0.5 : sudo dpkg --force-confold --force-all -i /home/caspida/<Extracted uba external package folder>/postgresql*.deb Start all UBA Services : /opt/caspida/bin/Caspida start-all
2022-12-05	UBA-16617	Repeated Kafka warning message "Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order" Workaround: 1) On zookeeper node (typically node 2 on a multi-node deployment), find all leader-epoch-checkpoint files: locate leader-epoch-checkpoint (can also use a find command if locate isn't available) a) Copy result into a script, adding ">" prior to each result. i.e. #!/bin/bash > /var/vcap/store/kafka/AnalyticsTopic-0/leader-epoch-checkpoint > /var/vcap/store/kafka/AnalyticsTopic-1/leader-epoch-checkpoint > /var/vcap/store/kafka/AnalyticsTopic-10/leader-epoch-checkpoint > /var/vcap/store/kafka/AnalyticsTopic-11/leader-epoch-checkpoint ... b) Make script executable: chmod +x <script name>.sh 2) On node 1, run: /opt/caspida/bin/Caspida stop-all 3) On zookeeper node, run: ./<script name>.sh 4) On node 1, run: /opt/caspida/bin/Caspida start-all 5) Check logs to see if warn messages still show up on zookeeper node: tail -f /var/vcap/sys/log/kafka/server.log 6) If you see the following warning repeated: WARN Resetting first dirty offset of __consumer_offsets-17 to log start offset 3346 since the checkpointed offset 3332 is invalid. (kafka.log.LogCleanerManager$) a) Clear cleaner-offset-checkpoint on zookeeper node by running: > /var/vcap/store/kafka/cleaner-offset-checkpoint b) Then on node 1, run: /opt/caspida/bin/Caspida stop-all && /opt/caspida/bin/Caspida start-all
2022-07-26	UBA-15997	Benign error messages on CaspidaCleanup: Relations do not exist, Kafka topic does not exist on ZK path
2022-02-14	UBA-15364	Spark HistoryServer running out of memory for large deployments with error: "java.lang.OutOfMemoryError: GC overhead limit exceeded" Workaround: Open the following file to edit on the Spark History Server: /var/vcap/packages/spark/conf/spark-env.sh You can check deployments.conf field spark.history to find out which node runs the Spark History Server. Update the following setting to 3G: SPARK_DAEMON_MEMORY=3G Afterwards, restart the spark services: /opt/caspida/bin/Caspida stop-spark && /opt/caspida/bin/Caspida start-spark
2021-08-30	UBA-14755	Replication.err logging multiple errors - Cannot delete snapshot s_new from path /user: the snapshot does not exist.
2020-04-07	UBA-13804	Kubernetes certificates expire after one year Workaround: Run the following commands on the Splunk UBA master node: /opt/caspida/bin/Caspida remove-containerization /opt/caspida/bin/Caspida setup-containerization /opt/caspida/bin/Caspida stop-all /opt/caspida/bin/Caspida start-all
2019-10-07	UBA-13227	Backend anomaly and custom model names are displayed in Splunk UBA Workaround: Click the reload button in the web browser to force reload the UI page.
2019-08-29	UBA-13020	Anomalies migrated from test-mode to active-mode won't be pushed to ES
2019-08-06	UBA-12910	Splunk Direct - Cloud Storage does not expose src_ip field Workaround: When ingesting Office 365 Sharepoint/OneDrive logs through Splunk Direct - Cloud Storage, add an additional field mapping for src_ip in the final SPL to be mapped from ClientIP (| eval src_ip=ClientIP). Make sure to add src_ip in the final list of fields selected using the fields command. For example: | fields app,change_type,dest_user,file_hash,file_size,object,object_path,object_type,parent_category,parent_hash,sourcetype,src_user,tag,src_ip