Docs » Supported integrations in Splunk Observability Cloud » Instrument back-end applications to send spans to Splunk APM » Instrument Java applications for Splunk Observability Cloud » Configure the Java agent for Splunk Observability Cloud

Configure the Java agent for Splunk Observability Cloud πŸ”—

You can configure the Java agent from the Splunk Distribution of OpenTelemetry Java to suit most of your instrumentation needs. In most cases, modifying the basic configuration is enough to get started. More advanced settings are also available.

The following sections describe all available settings for configuring the Java Virtual Machine (JVM) agent, including options for activating new features that are unique to the Splunk Distribution of OpenTelemetry Java.

Configuration methods πŸ”—

You can change the agent settings in two ways:

  • Set an environment variable. For example:

    export OTEL_SERVICE_NAME=my-java-app
  • Add a system property as runtime parameter. For example:

    java -javaagent:./splunk-otel-javaagent.jar \<my-java-app> \
    -jar <myapp>.jar

Environment variables are the preferred way of configuring OpenTelemetry agents. System properties, if specified, override existing environment variables.

General settings πŸ”—

The following settings are specific to the Splunk Distribution of OpenTelemetry Java:

Environment variable



The name of your organization’s realm, for example, us0. When you set the realm, telemetry is sent directly to the ingest endpoint of Splunk Observability Cloud, bypassing the Splunk Distribution of OpenTelemetry Collector.

System property: splunk.realm


A Splunk authentication token that lets exporters send data directly to Splunk Observability Cloud. Unset by default. Not required unless you need to send data to the Splunk Observability Cloud ingest endpoint. See Create and manage authentication tokens using Splunk Observability Cloud.

System property: splunk.access.token


Activates the addition of server trace information to HTTP response headers. For more information, see Server trace information. The default value is true.

System property: splunk.trace-response-header.enabled


Adds the full command line as a resource attribute for all metrics. If false, commands longer than 255 characters are truncated.

System property: splunk.metrics.force_full_commandline

Trace configuration πŸ”—

The following settings control tracing limits and attributes:

Environment variable



Name of the service or application you’re instrumenting. Takes precedence over the service name defined in the OTEL_RESOURCE_ATTRIBUTES variable.

System property:


Comma-separated list of resource attributes added to every reported span. For example, key1=val1,key2=val2.

System property: otel.resource.attributes


Used to add a peer.service attribute by specifying a comma-separated list of mapping from hostnames or IP addresses. For example, if set to,, requests to have a peer.service attribute of cats-service and requests to have one of dogs-api.

System property: otel.instrumentation.common


Adds @WithSpan annotation functionality for the target method string. Format is my.package.MyClass1[m1,m2];my.package.MyClass2[m3].

System property: otel.instrumentation.methods.include


Suppresses @WithSpan instrumentation for specific methods. Format is my.package.MyClass1[m1,m2];my.package.MyClass2[m3].

System property: otel.instrumentation.opentelemetry


Maximum number of attributes per span. The default value is unlimited.

System property: otel.span.attribute.count.limit


Maximum number of events per span. The default value is unlimited.

System property: otel.span.event.count.limit


Maximum number of links per span. The default value is 1000.

System property:

Exporters configuration πŸ”—

The following settings control trace exporters and their endpoints:

Environment variable



Trace exporter to use. You can set multiple comma-separated values.

System property: otel.traces.exporter


OTLP gRPC endpoint. The default value is http://localhost:4317.

System property: otel.exporter.otlp.endpoint

The Splunk Distribution of OpenTelemetry Java uses the OTLP gRPC span exporter by default. To send data directly to Splunk Observability Cloud, see Send data directly to Splunk Observability Cloud.

Samplers configuration πŸ”—

The following settings control trace sampling:

Environment variable



Sampler to use. The default value is always_on.

In addition to the samplers provided by the OpenTelemetry Java SDK, you can use the following samplers:

  • internal_root_off: Drops all traces with root spans where spanKind is INTERNAL, CLIENT or PRODUCER. Keeps root spans where spanKind is SERVER or CONSUMER.

  • rules: Drops all traces that originate from specific endpoints, as defined by the value of the OTEL_TRACES_SAMPLER_ARG setting. Only applies to spans where spanKind is SERVER.

System property: otel.traces.sampler


Semicolon-separated list of rules for the rules sampler. For example:


The following rules are supported:

  • drop=<value>: The sampler drops a span if its attribute has a substring equal to the value you’ve provided. For example: drop=/status.

  • fallback=<sampler>: Sampler to use if no drop rule matched a given span. Supported samplers are always_on and parentbased_always_on. If you define multiple fallback samplers, the Java agent uses the last one.

If you don’t set arguments when using the rules sampler, the instrumentation defaults to the parentbased_always_on sampler.

System property: otel.traces.sampler.arg

Example of trace sampling πŸ”—

The following example shows how to use the rules traces sampler to exclude the /healthcheck endpoint from monitoring:

export OTEL_TRACES_SAMPLER_ARG=drop=/healthcheck;fallback=parentbased_always_on

All requests to downstream services that happen as a consequence of calling an excluded endpoint are also excluded.

Propagators configuration πŸ”—

The following settings control trace propagation:

Environment variable



Comma-separated list of propagators you want to use. The default value is tracecontext,baggage. You can find the list of supported propagators in the OpenTelemetry documentation.

System property: otel.propagators

For backward compatibility with older versions of the Splunk Distribution of OpenTelemetry Java or the SignalFx Java Agent, use the b3multi trace propagator:

export OTEL_PROPAGATORS=b3multi

Java settings for AlwaysOn Profiling πŸ”—

The following settings control the AlwaysOn Profiling feature for the Java agent:

Environment variable



Activates AlwaysOn Profiling. The default value is false.

System property: splunk.profiler.enabled


The collector endpoint for profiler logs. By default, it takes the value of otel.exporter.otlp.endpoint.

System property: splunk.profiler.logs-endpoint


The location of the JDK Flight Recorder files. The default value is the local directory (.).

System property:


The duration of the recording unit. You can define duration in the form <number><unit>, where the unit can be ms, s, m, h, or d. The default interval is 20s. If you enter a number but not a unit, the default unit is assumed to be ms.

System property: splunk.profiler.recording.duration


Whether to preserve JDK Flight Recorder (JFR) files or not. The default value is false, which means that JFR files are deleted after processing.

System property: splunk.profiler.keep-files


Frequency with which call stacks are sampled, in milliseconds. The default value is 10000 milliseconds.

System property:


Activates memory profiling with all the options. To activate or deactivate specific memory profiling options, set their values explicitly.
  • The default value is false.

  • Requires splunk.profiler.enabled to be set to true.

  • Activating memory profiling sets the value of splunk.metrics.enabled to true.


OpenJDK versions 15.0 to 17.0.8, are not supported for memory profiling. See in the JDK bug system for more information.

System property: splunk.profiler.memory.enabled


Rate limit for memory profiling data, expressed as stack traces per unit of time. You can define duration in the form <number>/<unit>, where the unit can be s or m. The default value is 150/s, or 150 stack traces per second. Consider increasing this value when collecting memory profiling data from complex, multithreaded workloads, like application servers.

System property: splunk.profiler.memory.event.rate


Whether to activate TLAB memory events. The default value is the value assigned to the splunk.profiler.memory.enabled property.

System property: splunk.profiler.tlab.enabled


Whether to include stack traces of the agent internal threads and stack traces with JDK internal frames. The default value is false.

System property: splunk.profiler.include.internal.stacks


Whether to include only stack traces that are linked to a span context. The default value is false. When set to true, call stacks not linked to span contexts are dropped, which is useful to reduce data ingest volume.

System property: splunk.profiler.tracing.stacks.only

For more information on AlwaysOn Profiling, see Introduction to AlwaysOn Profiling for Splunk APM.

Metrics collection settings πŸ”—

The following settings control metrics collection for the Java agent:

Environment variable



Activates exporting metrics. See Metrics and attributes collected by the Splunk OTel Java agent for more information.
  • Default is false.

  • If you activate memory profiling using the splunk.profiler.memory.enabled property, the value of splunk.metrics.enabled is set to true.

System property: splunk.metrics.enabled


The OTel collector metrics endpoint. Default is http://localhost:9943/v2/datapoint.

System property: splunk.metrics.endpoint


Deprecated. Use OTEL_METRIC_EXPORT_INTERVAL instead. See Periodic exporting MetricReader in the official OpenTelemetry documentation.


Metric support is experimental.

Server trace information πŸ”—

To connect Real User Monitoring (RUM) requests from mobile and web applications with server trace data, trace response headers are activated by default. The instrumentation adds the following response headers to HTTP responses:

Access-Control-Expose-Headers: Server-Timing
Server-Timing: traceparent;desc="00-<serverTraceId>-<serverSpanId>-01"

The Server-Timing header contains the traceId and spanId parameters in traceparent format. For more information, see the Server-Timing and traceparent documentation on the W3C website.

The following server frameworks and libraries add Server-Timing information:

  • Servlet API versions 2.2 to 4.X.

  • Netty versions 3.8 to 4.0.


If you need to deactivate trace response headers, set SPLUNK_TRACE_RESPONSE_HEADER_ENABLED to false.

Other settings πŸ”—

Environment variable



Globally activates the Java agent automatic instrumentation. The default value is true. Useful for deactivating automatic discovery in testing scenarios or pipelines.

System property: otel.javaagent.enabled