Use the Splunk Universal Forwarder with the Collector 🔗
While Splunk Observability Cloud uses the OpenTelemetry Collector as the agent to capture traces, metrics, and logs, Splunk Enterprise Cloud uses the Splunk Universal Forwarder to capture logs and some metrics, which are also stored as logs. Learn more at Splunk Universal Forwarder (UF) .
Nevertheless, you can also deploy the Splunk Distribution of OpenTelemetry Collector alongside the Splunk Universal Forwarder (UF) on your virtual machines (VM) to manage your data ingestion, including logs, manually.
This solution is applicable for VM environments for operating systems that are currently supported by both Splunk Observability Cloud and Enterprise and Cloud, running in common environments such as AWS EC2, GCE, Azure VMs, and VMWare.
For Kubernetes deployments, use the Splunk Distribution of OpenTelemetry Collector for Kubernetes. Install the Collector using the method that best suits your needs:
The benefits of using the Universal Forwarder with the Collector are:
You can use Splunk Observability Cloud alongside Enterprise or Enterprise Cloud without capturing and submitting any duplicate telemetry data.
You do not have to update existing UF deployments.
Collect logs with the UF 🔗
The Collector can capture logs using Fluentd, but this option is deactivated by default. Alternatively, you can use the UF to send logs to Splunk Observability Cloud.
In Kubernetes environments, native OTel log collection is supported by default. See more at Configure logs and events for Kubernetes.
Collect data with the Collector and Universal Forwarder 🔗
To collect data with the Collector and the UF:
Configure each agent using the default configuration files:
Run the following command to install the Collector:
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \ sudo sh /tmp/splunk-otel-collector.sh --realm SPLUNK_REALM -- SPLUNK_ACCESS_TOKEN
Ensure that the UF captures the fully qualified domain name (FQDN) of the host, which is used to identify hosts in Splunk Observability Cloud. The UF can already capture this, and its behavior is consistent with the Collector. To capture the FQDN:
$SPLUNK_HOME/etc/system/local/directory, open server.conf and verify that the following stanza is present:
[general] hostnameOption = fullyqualifiedname
$SPLUNK_HOME/etc/system/local/ directorydirectory, open inputs.conf and verify that the following stanza is present:
Restart the UF.
Ensure that the UF captures the name of the service, which you must set manually in the Collector configuration and within your applications.
For the UF, do this in the same way that you append trace and span IDs to logs.
To capture the name of the service, set the
OTEL_SERVICE_NAMEenvironment variable in the configuration file. On Linux, run
export OTEL_SERVICE_NAME=<yourServiceName>. On Windows Powershell, run
$env:OTEL_SERVICE_NAME=<yourServiceName>. See https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#environment-variables on GitHub to view additional OpenTelemetry specification environment variables.
Restart both agents.