Docs » Get started with the Splunk Distribution of the OpenTelemetry Collector » Splunk OpenTelemetry Zero Configuration Auto Instrumentation » Splunk OpenTelemetry Zero Configuration Auto Instrumentation for Java » Zero Configuration Auto Instrumentation for Linux Java applications

Caution

Preview: Splunk Zero Config Auto Instrumentation

Preview features described in this document are provided by Splunk to you “as is” without any warranties, maintenance and support, or service-level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms .

Zero Configuration Auto Instrumentation for Linux Java applications 🔗

Zero Configuration Auto Instrumentation for Java activates automatic instrumentation for Java applications running on Linux. After installing the package, you must start or restart any Java applications that you want to instrument.

Prerequisites 🔗

Install the package 🔗

You can install the splunk-otel-auto-instrumentation package in the following ways:

To install the package, run the Collector installer script with the --with-instrumentation option. The installer script will install the Collector and the Java agent from the Splunk Distribution of OpenTelemetry Java. The Java agent is then loaded automatically when a Java application starts on the local machine.

Run the installer script with the --with-instrumentation option, as shown in the following example. Replace <SPLUNK_REALM> and <SPLUNK_ACCESS_TOKEN> with your Splunk Observability Cloud realm and token, respectively.

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN>

Note

If you have a Log Observer entitlement or wish to collect logs for the target host, make sure Fluentd is installed and enabled in your Collector instance.

To automatically define the optional deployment.environment resource attribute at installation time, run the installer script with the --deployment-environment <env> option. Replace <env> with the desired attribute value, for example, prod, as shown in the following example:

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --deployment-environment prod \
--realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN>

You can activate AlwaysOn Profiling for CPU and memory, as well as metrics, using additional options, as in the following example:

curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --with-instrumentation --deployment-environment prod \
--realm <SPLUNK_REALM> -- <SPLUNK_ACCESS_TOKEN> \
--enable-profiler --enable-profiler-memory --enable-metrics

Next, ensure the service is running and restart your application. See Ensure the service is running and Start your applications.

Ensure the service is running 🔗

After a successful installation, run the following command to ensure the splunk-otel-collector service is running:

sudo systemctl status splunk-otel-collector

If the service is not running, start or restart it with the following command:

sudo systemctl restart splunk-otel-collector

If the service fails to start, check that the SPLUNK_REALM and SPLUNK_ACCESS_TOKEN in /etc/otel/collector/splunk-otel-collector.conf are correct. You can also view the service logs with this command:

sudo journalctl -u splunk-otel-collector

Start your applications 🔗

For auto instrumentation to take effect, you must manually start or restart any Java applications on the host where you installed the package. This is true after installing the auto instrumentation package for the first time and whenever you make any changes to the configuration file.

After your applications are running, you can verify your data. See Verify that your data is coming into Splunk APM. You can also configure instrumentation settings. See (Optional) Configure the instrumentation.

(Optional) Configure the instrumentation 🔗

The default settings for zero config autoinstrumentation are sufficient for most cases. You can set additional customizations by editing the configuration file.

The installation package contains the following artifacts:

  • The configuration file at /usr/lib/splunk-instrumentation/instrumentation.conf

  • The Java Instrumentation Agent at /usr/lib/splunk-instrumentation/splunk-otel-javaagent.jar

  • The shared instrumentation library at /usr/lib/splunk-instrumentation/libsplunk.so`

The following is a sample of the default configuration file:

java_agent_jar=/usr/lib/splunk-instrumentation/splunk-otel-javaagent.jar

By default, the configuration file only specifies one parameter, java_agent_jar, which points to the path of the installed Java Instrumentation Agent.

The following is a sample configuration that sets the service name and environment, activates CPU and memory profiling, and activates metric collection:

java_agent_jar=/usr/lib/splunk-instrumentation/splunk-otel-javaagent.jar
service_name=default.service
resource_attributes=deployment.environment=test
enable_profiler=true
enable_profiler_memory=true
enable_metrics=true

The Supported parameters section discusses additional parameters you can set in the configuration file.

Supported parameters 🔗

The following table shows the supported parameters for the /usr/lib/splunk-instrumentation/instrumentation.conf file.

Parameter

Description

Required

java_agent_jar

The full path to the JAR file provided by the installer.

Yes

service_name

An optional parameter that specifies a unique identifier for a particular host. If you set this parameter, all instrumented Java applications on the host have their service name set using the OTEL_SERVICE_NAME environment variable. If this parameter isn’t set, the shared object assigns a generated name.

No

resource_attributes

Contains a comma-separated list of name-value pairs of the form name=value. Use this attribute to add extra tags to the generated trace data.
If you installed the package with the installer script and specified the --deployment-environment <your_env> when you ran the script, the deployment.environment=<your_env> resource attribute is automatically added to the configuration.

No

generate_service_name

Set to false to prevent the preloader from setting the OTEL_SERVICE_NAME environment variable. In that case, the instrumentation library tries to determine the service name automatically.

No

enable_profiler

Set to true to activate AlwaysOn Profiling (CPU). See Introduction to AlwaysOn Profiling for Splunk APM.

No

enable_profiler_memory

Set to true to activate AlwaysOn Profiling (Memory). See Memory profiling metrics.

No

enable_metrics

Set to true to activate metric collection. See Metrics and attributes collected by the Splunk OTel Java agent.

No

disable_telemetry

Set to true to prevent the preloader from sending the splunk.linux-autoinstr.executions metric to the local collector.

No

The /etc/ld.so.preload file is automatically created or updated with the default path to the installed instrumentation library. If necessary, custom library paths can be manually added to this file.

Note

Whenever you change the configuration file, you must manually start or restart any Java applications on the host where you installed the package.

Advanced configuration 🔗

More advanced configuration options like correlating traces and logs and activating custom sampling are available by configuring the Java agent.

Upgrade the package 🔗

You can upgrade the package in two ways:

Upgrade using the package repository 🔗

If you installed the package using the installer script, or if you configured the Debian or RPM package repositories manually, run the following commands according to your platform. Upgrading the package requires root privileges.

Debian 🔗

Run the following commands:

sudo apt-get update
sudo apt-get --only-upgrade splunk-otel-auto-instrumentation

You might see a prompt to keep or overwrite the configuration file at /usr/lib/splunk-instrumentation/instrumentation.conf. If you choose to overwrite, the configuration file reverts to the default file provided by the upgraded package.

RPM 🔗

For the RPM package management system, run the following commands:

yum 🔗
sudo yum upgrade splunk-otel-auto-instrumentation
dnf 🔗
sudo dnf upgrade splunk-otel-auto-instrumentation
zypper 🔗
sudo zypper refresh
sudo zypper update splunk-otel-auto-instrumentation

After you’ve upgraded the packages, manually start or restart the Java applications on the host for the changes to take effect.

Upgrade using Debian or RPM packages 🔗

To manually upgrade the package:

  1. Download the splunk-auto-auto-instrumentation Debian or RPM package for the target system from the GitHub Releases page .

  2. Run the following commands to install the package. Replace <path to splunk-otel-auto-instrumentation deb/rpm> with the local path to the downloaded package:

    sudo dpkg -i <path to splunk-otel-auto-instrumentation deb>

After upgrading the Debian package, you might see a prompt to keep or overwrite the configuration file at /usr/lib/splunk-instrumentation/instrumentation.conf. If you choose to overwrite, the configuration file reverts to the default file provided by the upgraded package.

You can also upgrade using the same package repositories as the Collector. See Debian or RPM packages for more information.

Deactivate automatic instrumentation 🔗

Use one of the following options to deactivate automatic instrumentation:

  • Uninstall the package by running the following command:

    curl -sSL https://dl.signalfx.com/splunk-otel-collector.sh > /tmp/splunk-otel-collector.sh && \
sudo sh /tmp/splunk-otel-collector.sh --uninstall

    See Uninstall on Linux for more information on the files deleted by the uninstall.

  • Set DISABLE_SPLUNK_AUTOINSTRUMENTATION to any nonempty value other than false, FALSE, or 0.

  • Set the JAVA_TOOL_OPTIONS environment variable to some value that you want the JVM to pick up.

  • Delete or move the instrumentation.conf configuration file.

Next steps 🔗

After activating automatic instrumentation, Verify that your data is coming into Splunk APM.

Troubleshooting 🔗

If you activate auto instrumentation and you see an error message or you do not see any data in Observabiity Cloud APM, try the following steps:

  • Check the splunk-otel-collector service logs:

sudo journalctl -u splunk-otel-collector

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways.

Available to Splunk Observability Cloud customers

Available to prospective customers and free trial users

  • Ask a question and get answers through community support at Splunk Answers .

  • Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide. To join, see Chat groups in the Get Started with Splunk Community manual.

To learn about even more support options, see Splunk Customer Success .