Docs » Get started with the Splunk Distribution of the OpenTelemetry Collector » Discover telemetry sources automatically » Automatic discovery and configuration for Linux » Automatic discovery and configuration for back-end applications in Linux

Automatic discovery and configuration for back-end applications in Linux 🔗

When using automatic discovery and configuration, the Splunk Distribution of OpenTelemetry Collector automatically detects back-end applications running in your Linux environment.

By deploying the Collector with automatic discovery, you can instrument applications and send data to Splunk Observability Cloud without editing your application’s code or configuring files.

Automatic discovery for Linux can detect and configure the following applications and language runtimes:

  • Java

  • Node.js

  • .NET

How automatic discovery for Linux works 🔗

Automatic discovery for Linux operates as a mode of the Splunk Distribution of OpenTelemetry Collector. You install and activate automatic discovery for the Collector by using the Linux installer script or package manager. During installation, you can specify the types of language runtimes you want the Collector to detect.

After installation, the Collector runs in your Linux environment and listens for requests to your applications. When the Collector detects activity, it gathers telemetry data from your application runtime and sends this data to Splunk Application Performance Monitoring (APM).

Requirements 🔗

You need the following components to use automatic discovery for back-end Linux applications:

  • systemd

  • curl

  • sudo

Make sure you’ve also installed the components specific to your language runtime:

Java version 8 or higher and supported libraries. See Java agent compatibility and requirements for more information.

Get started 🔗

To install and use automatic discovery for Linux, follow these steps:

  1. Install the package

  2. Ensure the collector service is running

  3. View results in Splunk APM

Install the package 🔗

Using the installer script, you can install and activate automatic discovery for either all supported applications on the host via the system-wide method or only for applications running as systemd services.

Using the installer script, you can install the automatic discovery package for Java and activate automatic discovery for Java for either all supported Java applications on the host via the system-wide method or for only Java applications running as systemd services.

Note

By default, automatic discovery is activated for all languages (Java, Node.js, and .NET) when using the installer script. To deactivate automatic discovery for other languages, add the --without-instrumentation-sdk [language] option in the installer script command.

Run the installer script with the --with-instrumentation option, as shown in the following example. Replace <SPLUNK_REALM> and <SPLUNK_ACCESS_TOKEN> with your Splunk Observability Cloud realm and token, respectively.

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN>

Note

If you wish to collect logs for the target host, make sure Fluentd is installed and enabled in your Collector instance by specifying the --with-fluentd option.

The system-wide automatic discovery method automatically adds environment variables to /etc/splunk/zeroconfig/java.conf.

To automatically define the optional deployment.environment resource attribute at installation time, run the installer script with the --deployment-environment <env> option. Replace <env> with the desired attribute value, for example, prod, as shown in the following example:

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --deployment-environment prod \
--realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN>

You can activate AlwaysOn Profiling for CPU and memory, as well as metrics, using additional options, as in the following example:

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --deployment-environment prod \
--realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN> \
--enable-profiler --enable-profiler-memory --enable-metrics

Next, ensure the service is running and restart your application. See Ensure the collector service is running and Start your applications.

Ensure the collector service is running 🔗

After a successful installation, run the following command to ensure the splunk-otel-collector service is running:

sudo systemctl status splunk-otel-collector

If the service is not running, start or restart it with the following command:

sudo systemctl restart splunk-otel-collector

If the service fails to start, check that the SPLUNK_REALM and SPLUNK_ACCESS_TOKEN in /etc/otel/collector/splunk-otel-collector.conf are correct. You can also view the service logs with this command:

sudo journalctl -u splunk-otel-collector

Start your applications 🔗

For automatic discovery to take effect, you must either restart the host or manually start or restart any applications on the host where you installed the package. You must restart the host or applications after installing the automatic discovery package for the first time and whenever you make any changes to the configuration file.

After your applications are running, you can verify your data. See View results in Splunk APM. You can also configure instrumentation settings. See (Optional) Configure the instrumentation.

(Optional) Configure the instrumentation 🔗

You can configure the Splunk Distribution of OpenTelemetry Collector to suit your instrumentation needs. In most cases, modifying the basic configuration is enough to get started.

To learn more, see the following resources:

Update automatic discovery and configuration 🔗

You can upgrade the package by using the package repository or by using Debian or RPM packages.

If you installed the package using the installer script, or if you configured the Debian or RPM package repositories manually, run the following commands according to your platform. Upgrading the package requires root privileges.

Run the following commands:

sudo apt-get update
sudo apt-get --only-upgrade splunk-otel-auto-instrumentation

You might see a prompt to keep or overwrite the configuration file(s) in /etc/splunk/zeroconfig/. If you choose to overwrite, the configuration file reverts to the default file provided by the upgraded package.

Troubleshooting 🔗

To troubleshoot common errors that occur when instrumenting applications, see the following troubleshooting guides:

View results in Splunk APM 🔗

After activating automatic discovery, ensure your data is flowing into Splunk Observability Cloud. See Verify that your data is coming into Splunk APM.

This page was last updated on Oct 09, 2024.