Install the Collector for Linux with the installer script π
The Splunk Distribution of OpenTelemetry Collector for Linux is a package that provides integrated collection and forwarding for all data types.
Install the package using one of these methods:
Installer script π
The following Linux distributions and versions are supported:
Amazon Linux: 2, 2023. Log collection with Fluentd is not currently supported for Amazon Linux 2023.
CentOS, Red Hat, or Oracle: 7, 8, 9
Debian: 9, 10, 11
SUSE: 12, 15 for version 0.34.0 or higher. Log collection with Fluentd is not currently supported.
Ubuntu: 16.04, 18.04, 20.04, and 22.04
The installer script deploys and configures these elements:
The Splunk Distribution of OpenTelemetry Collector for Linux
Fluentd, using the td-agent. Turned off by default. See Fluent Forward receiver for more information
To install the package using the installer script, follow these steps:
Ensure you have systemd,
curl
andsudo
installed.Download and run the installer script.
Replace the following variables for your environment:
SPLUNK_REALM
: This is the Realm to send data to. The default isus0
. See realms .SPLUNK_MEMORY_TOTAL_MIB
: This is the total allocated memory in mebibytes (MiB). For example,512
allocates 512 MiB (500 x 2^20 bytes) of memory.SPLUNK_ACCESS_TOKEN
: This is the base64-encoded access token for authenticating data ingest requests. See Create and manage organization access tokens using Splunk Observability Cloud.
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh;
sudo sh /tmp/splunk-otel-collector.sh --realm $SPLUNK_REALM --memory $SPLUNK_MEMORY_TOTAL_MIB -- $SPLUNK_ACCESS_TOKEN
Note
If you have a Log Observer entitlement or want to collect logs for the target host with Fluentd, use the --with-fluentd
option to also install Fluentd when installing the Collector.
Configure memory allocation π
To configure memory allocation, change the --memory
parameter. By default, this parameter is set to 512 MiB, or 500 x 2^20 bytes, of memory. Increase this setting to allocate more memory, as shown in the following example.
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh;
sudo sh /tmp/splunk-otel-collector.sh --realm $SPLUNK_REALM --memory $SPLUNK_MEMORY_TOTAL_MIB \
-- $SPLUNK_ACCESS_TOKEN
Configure proxy settings π
To configure proxy settings to install and run the OpenTelemetry Collector, see Configure proxy settings for the Collector.
Use configured repos π
By default, apt/yum/zypper repo definition files are created to download the package and Fluentd deb/rpm packages from https://splunk.jfrog.io/splunk and https://packages.treasuredata.com , respectively.
To skip these steps and use configured repos on the target system that provide the splunk-otel-collector
and td-agent
deb/rpm packages, specify the --skip-collector-repo
or --skip-fluentd-repo
options. For example:
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --realm $SPLUNK_REALM --skip-collector-repo --skip-fluentd-repo \
-- $SPLUNK_ACCESS_TOKEN
Configure Fluentd π
Fluentd is turned off by default. To install Fluentd for log collection, run the installer script with the --with-fluentd
option. For example:
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-fluentd --realm $SPLUNK_REALM -- $SPLUNK_ACCESS_TOKEN
When turned on, the Fluentd service is configured by default to collect and forward log events with the @SPLUNK
label to the Collector, which then sends these events to the HEC ingest endpoint determined by the --realm <SPLUNK_REALM>
option. For example, https://ingest.<SPLUNK_REALM>.signalfx.com/v1/log
.
The following Fluentd plugins are also installed:
capng_c
for activating Linux capabilities.fluent-plugin-systemd
for systemd journal log collection.
Additionally, the following dependencies are installed as prerequisites for the Fluentd plugins:
Debian-based systems:
build-essential
libcap-ng0
libcap-ng-dev
pkg-config
RPM-based systems:
Development Tools
libcap-ng
libcap-ng-devel
pkgconfig
You can specify the following parameters to configure the package to send log events to a custom Splunk HTTP Event Collector (HEC) endpoint URL:
--hec-url <URL>
--hec-token <TOKEN>
HEC lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. See Set up and use HTTP Event Collector in Splunk Web <https://docs.splunk.com/Documentation/Splunk/8.2.1/Data/UsetheHTTPEventCollector>.
The main Fluentd configuration is installed to /etc/otel/collector/fluentd/fluent.conf
. Custom Fluentd source configuration files can be added to the /etc/otel/collector/fluentd/conf.d
directory after installation.
Note the following:
In this directory, all files with the .conf extension are automatically included by Fluentd.
The td-agent user must have permissions to access the configuration files and the paths defined within.
By default, Fluentd is configured to collect systemd journal log events from
/var/log/journal
.
After any configuration modification, run sudo systemctl restart td-agent
to restart the td-agent service.
If the td-agent package is upgraded after initial installation, you might need to set the Linux capabilities for the new version by performing the following steps for td-agent versions 4.1 or higher:
Check for the activated capabilities:
sudo /opt/td-agent/bin/fluent-cap-ctl --get -f /opt/td-agent/bin/ruby Capabilities in `` /opt/td-agent/bin/ruby`` , Effective: dac_override, dac_read_search Inheritable: dac_override, dac_read_search Permitted: dac_override, dac_read_search
If the output from the previous command does not include
dac_override
anddac_read_search
as shown above, run the following commands:sudo td-agent-gem install capng_c sudo /opt/td-agent/bin/fluent-cap-ctl --add "dac_override,dac_read_search" -f /opt/td-agent/bin/ruby sudo systemctl daemon-reload sudo systemctl restart td-agent
If you already installed Fluentd on a host, install the Collector without Fluentd using the --without-fluentd
option. For more information, see Configure the Collector.
Configure automatic instrumentation for Java π
You can also automatically instrument your Java applications along with the Collector installation. Auto instrumentation removes the need to install and configure the Java agent separately. See Splunk OpenTelemetry Zero Configuration Auto Instrumentation for Java for the installation instructions. For more information on Java instrumentation, see Instrument Java applications for Splunk Observability Cloud.
Options of the installer script for Linux π
The Linux installer script supports the following options:
Option |
Description |
Default value |
---|---|---|
|
Set the API endpoint URL explicitly instead of using the endpoint inferred from the specified realm. |
|
|
Set the ballast size explicitly instead of the value calculated using the |
|
|
Use the beta package repository. |
Not applicable |
|
Set the path to an existing custom configuration file for the Collector service instead of using the default configuration file provided by the Collector package based on the |
|
|
The Collector package version to install. |
|
|
Activate discovery mode on Collector startup. See Discover and configure metrics sources automatically for more information. |
Deactivated |
|
Set the HEC token if it`` s different than the specified |
|
|
Set the HEC endpoint URL explicitly instead of using the endpoint inferred from the specified realm. |
|
|
Set the ingest endpoint URL explicitly instead of using the endpoint inferred from the specified realm. |
|
|
Total memory in MIB to allocate to the Collector. This option automatically calculates the ballast size. See Sizing and scaling for more information. |
|
|
Configure the Collector service to run in host monitoring ( |
|
|
Network interface the Collector receivers listen on. |
|
|
The Splunk realm to use. The ingest, API, trace, and HEC endpoint URLs are automatically generated using this value. |
|
|
Set the group for the splunk-otel-collector service. The option creates the group if it doesnβt exist. |
|
|
Set the user for the splunk-otel-collector service. The option creates the user if it doesnβt exist. |
|
|
By default, the scripts create an apt, yum, or zypper repo definition file to download the Collector package from |
|
|
By default, the scripts create an apt, yum, or zypper repo definition file to download the fluentd package from |
|
|
Use the test package repo. |
Not applicable |
|
Set the trace endpoint URL explicitly instead of the endpoint inferred from the specified realm. |
|
|
Removes the Splunk OpenTelemetry Collector for Linux. |
Not applicable |
|
Whether to install and configure fluentd to forward log events to the Collector. See Configure Fluentd for more information. |
|
|
Whether to install and configure the splunk-otel-auto-instrumentation package. See Splunk OpenTelemetry Zero Configuration Auto Instrumentation for more information. |
|
|
Set the |
Empty |
|
Override the autogenerated service names for all instrumented Java applications on this host with |
Empty |
|
Specify |
|
|
Activate or deactivate the instrumentation preloader from sending the |
|
|
Activate or deactivate AlwaysOn CPU Profiling. Only applicable if the |
|
|
Activate or deactivate AlwaysOn Memory Profiling. Only applicable if the |
|
|
Activate or deactivate exporting Micrometer metrics. Only applicable if the |
|
|
The package version to install. Only applicable if the |
|
|
Use |
Not applicable |
To display all the configuration options supported by the script, use the -h
flag.
curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh;
sh /tmp/splunk-otel-collector.sh -h
Install the Collector using deployment tools π
Splunk offers the configuration management options described in this section.
Amazon ECS EC2 π
Note
Available for Prometheus only.
Splunk provides a task definition to deploy the Splunk Distribution of OpenTelemetry Collector to ECS EC2. The task definition is a text file, in JSON format, that describes one or more containers that form your application. See Amazon ECS EC2 for the installation instructions.
Amazon Fargate π
Note
Available for Prometheus only. Not yet available for Amazon EKS.
Splunk provides a guided setup to deploy the Splunk Distribution of OpenTelemetry Collector on Amazon Fargate as a sidecar (additional container) to Amazon ECS tasks. See Amazon Fargate for the installation instructions.
Ansible π
Splunk provides an Ansible role that installs the package configured to collect data (metrics, traces, and logs) from Linux machines and send that data to Splunk Observability Cloud. See Ansible for Linux for the instructions to download and customize the role.
Chef π
Splunk provides a cookbook to install the Collector using Chef. See Chef for the installation instructions.
Heroku π
The Splunk Distribution of OpenTelemetry Collector for Heroku is a buildpack for the Collector. The buildpack installs and runs the Collector on a Dyno to receive, process, and export metric and trace data for Splunk Observability Cloud. See Heroku for the steps to install the buildpack.
Nomad π
Use Nomad to deploy the Collector. See Nomad for the installation instructions.
Pivotal Cloud Foundry π
You can use one of these three options to deploy the Collector with Pivotal Cloud Foundry (PCF):
Collector standalone deployment.
Collector as a sidecar to your app.
Tanzu Tile.
See more in Pivotal Cloud Foundry.
Puppet π
Splunk provides a Puppet module to install and configure the package. A module is a collection of resources, classes, files, definition, and templates. See Puppet for Linux for the instructions to download and customize the module.
Salt π
Splunk provides a Salt formula to install and configure the Collector. See Salt for the instructions.
Next steps π
After you`` ve installed the package, you can perform these actions:
Use Infrastructure Monitoring to track the health of your infrastructure.
Use APM to monitor the performance of applications.
Use Log Observer Connect to analyze log events and troubleshoot issues with your services.