Metrics and attributes collected by the Splunk OTel Java agent 🔗

The agent of the Splunk Distribution of OpenTelemetry Java collects the following application metrics data and attributes in addition to all that the upstream OpenTelemetry agent collects. To learn about the different metric types, see Metric types.

Caution

OpenTelemetry Java Instrumentation 2.x contains a set of breaking changes, introduced as part of recent OpenTelemetry HTTP semantic convention updates. To migrate, see Migration guide for OpenTelemetry Java 2.x metrics.

Application metrics 🔗

The agent of the Splunk Distribution of OpenTelemetry Java collects the following application metrics.

Default metric dimensions 🔗

The following dimensions are automatically added to all metrics exported by the agent:

Dimension	Description
`deployment.environment`	Value of the `deployment.environment` resource attribute, if present.
`runtime`	Value of the `process.runtime.name` resource attribute, for example `OpenJDK Runtime Environment`.
`process.pid`	The Java process identifier (PID).
`container.id`	Identifier of the container, if applicable.
`host.name`	Name of the host.
`service`	Value of the `service.name` resource attribute.

Supported libraries 🔗

The agent collects the following metrics through the following libraries:

Library/Framework	Instrumentation	Supported versions
JVM metrics	`jvm-metrics-splunk`	Java runtimes version 8 and higher
Apache DBCP2 connection pool metrics	`commons-dbcp2-splunk`	Version 2.0 and higher
c3p0 connection pool metrics	`c3p0-splunk`	Version 0.9.5 and higher
HikariCP connection pool metrics	`hikaricp-splunk`	Version 3.0 and higher
Oracle Universal Connection Pool metrics (UCP)	`oracle-ucp-splunk`	Version 11.2.0.4 and higher
Tomcat JDBC connection pool metrics	`tomcat-jdbc-splunk`	Version 8.5 and higher
Vibur DBCP connection pool metrics	`vibur-dbcp-splunk`	Version 20.0 and higher

JVM metrics 🔗

The Splunk OTel Java agent collects the following Java Virtual Machine (JVM) metrics when metric collection is activated.

ClassLoader metrics 🔗

The agent collects the following ClassLoader metrics:

Metric	Type	Description
`jvm.class.count`	UpDown counter	Number of loaded classes.
`jvm.class.unloaded`	Counter	Total number of unloaded classes since the process started.

Garbage collection metrics 🔗

The agent collects the following garbage collection (GC) metrics:

Metric	Type	Description
`jvm.gc.duration{jvm.gc.name=<concurrent gcs>}`	Histogram	Time spent in concurrent phase, in milliseconds.
`jvm.memory.allocated`	Counter	Increase in the size of the young heap memory pool after 1 garbage collection and before the next.

Memory metrics 🔗

The agent collects the following memory metrics:

Metric	Type	Description
`jvm.memory.allocated`	Counter	Total number of bytes allocated by JVM threads since the previous data point was emitted. Use the rate per second rollup.
`jvm.buffer.count`	UpDown counter	An estimate of the number of buffers in the pool.
`jvm.buffer.memory.usage`	UpDown counter	An estimate of the memory that the JVM is using for this buffer pool, in bytes.
`jvm.buffer.memory.limit`	UpDown counter	An estimate of the total capacity of the buffers in this pool, in bytes.
`jvm.memory.committed`	UpDown counter	Amount of memory available to the JVM, in bytes.
`jvm.memory.limit`	UpDown counter	Maximum amount of memory available for memory management, in bytes.
`jvm.memory.used`	UpDown counter	Amount of used memory, in bytes.

Thread metrics 🔗

The agent collects the following thread metrics:

Metric	Type	Description
`jvm.thread.count`	UpDown counter	Number of live threads, including daemon and nondaemon threads.

Connection pool metrics 🔗

The Splunk Distribution of OpenTelemetry Java instruments several Java Database Connectivity (JDBC) connection pool implementations:

Apache DBCP2
c3p0
HikariCP
Oracle Universal Connection Pool (UCP)
Tomcat JDBC
Vibur DBCP

Each of the connection pools reports a subset of the following metrics:

Metric	Type	Description
`db.client.connections.usage[state=used]`	Gauge	Number of open connections that are in use.
`db.client.connections.usage[state=idle]`	Gauge	Number of open connections that are idle.
`db.client.connections.idle.max`	Gauge	Maximum number of idle open connections allowed.
`db.client.connections.idle.min`	Gauge	Minimum number of idle open connections allowed.
`db.client.connections.max`	Gauge	Maximum number of open connections allowed.
`db.client.connections.pending_requests`	Gauge	Number of threads that are waiting for an open connection.
`db.client.connections.timeouts`	Counter	Number of connection timeouts that have happened since the application started.
`db.client.connections.create_time`	Histogram	Time it took to create a new connection, in milliseconds.
`db.client.connections.wait_time`	Histogram	Time it took to get an open connection from the pool, in milliseconds.
`db.client.connections.use_time`	Histogram	Time between borrowing a connection and returning it to the pool, in milliseconds.

WebEngine attributes 🔗

The Splunk Distribution of OpenTelemetry Java captures data about the application server and adds the following attributes to SERVER spans:

Span attribute	Description
`webengine.name`	Name of the applications server. For example, `tomcat`.
`webengine.version`	Version of the application server.

For a list of supported application servers, see the Supported libraries and frameworks.

New metric names 🔗

The following table shows the metrics produced by default by OpenTelemetry Java 2.0 and higher, together with their legacy equivalent from version 1.x.

OTel Java 2.0 metric	Legacy metric (1.x)
`db.client.connections.create_time` (Histogram, ms)	`db.pool.connections.create_time`
`db.client.connections.idle.max`	`db.pool.connections.idle.max`
`db.client.connections.idle.min`	`db.pool.connections.idle.min`
`db.client.connections.max`	`db.pool.connections.max`
`db.client.connections.pending_requests`	`db.pool.connections.pending_threads`
`db.client.connections.timeouts`	`db.pool.connections.timeouts`
`db.client.connections.usage[state=idle]`	`db.pool.connections.idle`
`db.client.connections.usage[state=used]`	`db.pool.connections.active`
`db.client.connections.use_time` (Histogram, ms)	`db.pool.connections.use_time`
`db.client.connections.wait_time` (Histogram, ms)	`db.pool.connections.wait_time`
`jvm.buffer.count`	`runtime.jvm.buffer.count`
`jvm.buffer.memory.limit`	`runtime.jvm.buffer.total.capacity`
`jvm.buffer.memory.usage`	`runtime.jvm.buffer.memory.used`
`jvm.class.count`	`runtime.jvm.classes.loaded`
`jvm.class.unloaded`	`runtime.jvm.classes.unloaded`
`jvm.gc.duration{jvm.gc.name=<concurrent gcs>}` (Histogram) `jvm.gc.action`	`runtime.jvm.gc.concurrent.phase.time`
`jvm.gc.duration{jvm.gc.name!=<concurrent gcs>}` `jvm.gc.action`	`runtime.jvm.gc.pause`
`jvm.memory.allocated`^*	`runtime.jvm.gc.memory.allocated` `process.runtime.jvm.memory.allocated`
`jvm.memory.committed`	`runtime.jvm.memory.committed`
`jvm.memory.limit`	`runtime.jvm.memory.max`
`jvm.memory.limit{jvm.memory.pool.name=<long lived pools>}`	`runtime.jvm.gc.max.data.size`
`jvm.memory.used`	`runtime.jvm.memory.used`
`jvm.memory.used_after_last_gc{jvm.memory.pool.name=<long lived pools>}`	`runtime.jvm.gc.live.data.size`
`jvm.thread.count`	`runtime.jvm.threads.daemon` `runtime.jvm.threads.live`

^* This is a Splunk specific metric and it’s not present in the upstream semantic conventions.

Note

The previous table contains metrics generated by default. Additional metrics might be emitted by supported metrics instrumentation, for example when instrumenting application servers.

For more information on the HTTP semantic convention changes, see HTTP semantic convention stability migration guide on GitHub.

Deactivate metrics export 🔗

To turn off logs export to Splunk Observability Cloud, set the OTEL_METRICS_EXPORTER environment variable or the otel.metrics.exporter system property to none.

This page was last updated on Jul 08, 2024.

Related Topics