Splunk® Universal Forwarder

Forwarder Manual

This documentation does not apply to the most recent version of Splunk® Universal Forwarder. For documentation on the most recent version, go to the latest release.

Known issues

This topic lists known issues that are specific to the universal forwarder. For information on fixed issues, see Fixed issues.

Least privileged mode permissions issues

Least privilege mode is enabled read any file permission on both Linux version 9.0.0 ad later, and on Windows version 9.1.0 and later. A non-root or non-admin user that could not access some files before upgrade to least privilege user, may be able to access those files after upgrade in the following situations:

  • You perform a CX upgrade UF from old versions to least privilege version.
  • Before upgrade, your universal forwarder is running as non-root or non-local admin.
  • Prior to upgrade, you have inputs to monitor a directory with many files, or inputs with scripts to read many files, where users have no permission to access those files

This can lead to:

  • Security issues.
  • Performance issues: since the universal forwarder is able to read far more files than before, more resources are consumed.

To mitigate this issue, disable the "read any file" capability manually:

  • On Linux, edit the unit file to remove the CAP_DAC_READ_SEARCH capability. See Manage a Linux least-privileged user
  • On Windows, remove the SeBackupPrivilege capability from Windows local security policy. See your Microsoft documentation for more information.

Universal forwarder issues

Date filed Issue number Description
2022-12-01 SPL-233535, SPL-231086 UF 9.x Unnecessary user creation during silent installation

Delete etc/passwd manually after installation, create user-seed.conf with new user in SPLUNK_HOME/etc/system/local and restart Splunk

Customer claims that they were able to bypass the issue by using SPLUNKPASSWORD="" option during installation but I was not able to reproduce this example of user-seed.conf: https://drive.google.com/file/d/1GrypUL6719V0tGbw0Mv-u76jUYZ8LBS_/view?usp=sharing

2022-10-14 SPL-231514, SPL-228406 UF crash on EventLoop::run assert rv > 0

2022-09-08 SPL-229853, SPL-229208 PowerShell Modular input stopped working after UF 9.0 upgrade
2022-07-30 SPL-227653, SPL-231927 UF throws erroneous WARN for KVSTORE SSL misconfiguration on startup - server.conf//sslVerifyServerCert or "Starting migrate-kvstore."

It's safe to ignore the warning or you can disable the kvstore explicitly with server.conf:
disabled = true
2022-07-13 SPL-226795, SPL-222481, SPL-231443 Splunk UF Windows Event Log Stopped Being Ingested
2022-06-23 SPL-226019 Warning appears in the universal forwarder whenever any spl command is run: Warning: Attempting to revert the SPLUNK_HOME ownership Warning: Executing "chown -R splunk /opt/splunkforwarder". This warning is expected and will not affect functionality.
2022-06-22 SPL-226003, SPL-237740 When forwarding from an 9.0 instance with useAck enabled, ingestion stops after some time with errors: "Invalid ACK received from indexer="

As a workaround, disable useAck in outputs.conf on the forwarder. After disabling, indexers start to ingest data.

If you need useACK to prevent data loss, disabling autoBatch in outputs.conf can remediate the issue too, but it impacts throughput - no worse than 8.x, but no improvement for 9.0.

2022-06-06 SPL-225379 Ownership of files mentioned in manifest file is splunk:splunk instead of root:root after enabling boot start as root user for initd

When changing UF user, manually chown SPLUNK_HOME to the new user, including first time install/upgrade, or manually enable boot-start.
2022-05-16 SPL-224264, SPL-224265 Splunk UF not starting on Debian 11 (x86_64 and arm64)
2022-05-13 SPL-224167 Splunk UF for CentOS-7 (ARM64) is not available

UF for CentOS7 ARM 64 will be available in the 9.0.1 maintenance release.
2022-04-20 SPL-222917, SPL-230428 Crash in indexer discovery service on search head
2020-11-09 SPL-197140, SPL-234386 UF failed to start on Solaris 11.3 with error: "symbol in6addr_any: referenced symbol not found"

1. Do not upgrade past Splunk 8.0.5 on Solaris 11.3

OR 2. Upgrade to Solaris 11.4

Last modified on 25 October, 2023
Troubleshoot the universal forwarder   Fixed issues

This documentation applies to the following versions of Splunk® Universal Forwarder: 9.0.0

Was this topic useful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters