GenericJMX π
The Splunk Distribution of OpenTelemetry Collector uses the Smart Agent receiver with the genericjmx
monitor to expose metrics on Java Management Extensions (JMX), a generic framework to provide and query management information. The interface is used by the Java Virtual Machine (JVM) to provide information about the memory used and threads. For a more flexible alternative, use the JMX monitor.
This integration is only available on Kubernetes and Linux.
Benefits π
After you configure the integration, you can access these features:
View metrics. You can create your own custom dashboards, and most monitors provide built-in dashboards as well. For information about dashboards, see View dashboards in Observability Cloud.
View a data-driven visualization of the physical servers, virtual machines, AWS instances, and other resources in your environment that are visible to Infrastructure Monitoring. For information about navigators, see Splunk Infrastructure Monitoring navigators.
Access the Metric Finder and search for metrics sent by the monitor. For information, see Use the Metric Finder.
Installation π
Follow these steps to deploy this integration:
Deploy the Splunk Distribution of OpenTelemetry Collector to your host or container platform:
Configure the monitor, as described in the Configuration section.
Restart the Splunk Distribution of OpenTelemetry Collector.
Configuration π
To use this integration of a Smart Agent monitor with the Collector:
Include the Smart Agent receiver in your configuration file.
Add the monitor type to the Collector configuration, both in the receiver and pipelines sections.
Read more on how to Use Smart Agent monitors with the Collector.
See how to set up the Smart Agent receiver.
Learn about config options in Collector default configuration.
Example π
To activate this integration, add the following to your Collector configuration:
receivers:
smartagent/genericjmx:
type: collectd/genericjmx
... # Additional config
Next, add the monitor to the service.pipelines.metrics.receivers
section of your configuration file:
service:
pipelines:
metrics:
receivers: [smartagent/genericjmx]
Configuration settings π
The following table shows the configuration options for this monitor:
Option |
Required |
Type |
Description |
---|---|---|---|
|
yes |
|
The host to connect to. JMX must be configured for remote access and accessible from the agent. |
|
yes |
|
JMX connection port (not the RMI port) on the application. This corresponds to the |
|
no |
|
|
|
no |
|
This is how the service type is identified in the Splunk Observability Cloud UI so that you can get built-in content. |
|
no |
|
The JMX connection string. This is rendered as a Go template and has access to the other values in this config. Note: Do not set this string directly; setting the host and port as specified above is preferred. The default value is |
|
no |
|
Prefixes the generated plugin instance with prefix. If a second |
|
no |
|
Username to authenticate to the server |
|
no |
|
User password to authenticate to the server |
|
no |
|
Takes in key-value pairs of custom dimensions at the connection level. |
|
no |
|
A list of the MBeans defined in |
|
no |
|
A list of the MBeans to omit. This is useful when only a few MBeans need to be omitted from the default list. |
|
no |
|
Specifies how to map JMX MBean values to metrics. Specific service monitors such as Cassandra, Kafka, or ActiveMQ are pre-loaded with a set of mappings, and any that you add in this option will be merged with those. See GenericJMX plugin for more details. |
The nested mBeanDefinitions
configuration object has the following fields:
Option |
Required |
Type |
Description |
---|---|---|---|
|
no |
|
Sets the pattern which is used to retrieve MBeans from the MBeanServer. If more than one MBean is returned, you should use the |
|
no |
|
Prefixes the generated plugin instance with prefix. |
|
no |
|
The object names used by JMX to identify MBeans include βpropertiesβ, which are basically key-value pairs. If the given object name is not unique and multiple MBeans are returned, the values of those properties usually differ. You can use this option to build the plugin instance from the appropriate property values. This is optional and can be repeated to generate the plugin instance from multiple property values. |
|
no |
|
The |
|
no |
|
The nested values
configuration object has the following fields:
Option |
Required |
Type |
Description |
---|---|---|---|
|
no |
|
Sets the data set being used to handle the values of the MBean attribute. |
|
no |
|
Set to |
|
no |
|
Works like the option of the same name directly beneath the MBean block, but sets the type instance instead. |
|
no |
|
Works like the option of the same name directly beneath the MBean block, but sets the type instance instead. |
|
no |
|
Sets the name of the attribute from which to read the value. You can access the keys of composite types by using a dot to concatenate the key name to the attribute name. For example, βattrib0.key42β. If |
|
no |
|
The plural form of the |
Metrics π
Notes π
Learn more about the available metric types in Observability Cloud.
Default metrics are those metrics included in host-based subscriptions in Observability Cloud, such as host, container, or bundled metrics. Custom metrics are not provided by default and might be subject to charges. See more about metric categories.
To add additional metrics, see how to configure
extraMetrics
using the Collector.
Troubleshooting π
If you are not able to see your data in Splunk Observability Cloud, try these tips:
Submit a case in the Splunk Support Portal
Available to Splunk Observability Cloud customers
-
Available to Splunk Observability Cloud customers
Ask a question and get answers through community support at Splunk Answers
Available to Splunk Observability Cloud customers and free trial users
Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide
Available to Splunk Observability Cloud customers and free trial users
To learn how to join, see Get Started with Splunk Community - Chat groups
To learn about even more support options, see Splunk Customer Success.
Exposed ports π
The following Java properties show how to expose JMX ports to inbound connections. For more information, see Monitoring and Management Using JMX Technology in the Java documentation.
java \
-Dcom.sun.management.jmxremote.port=5000 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.rmi.port=5000 \
...
This works as long as the agent is allowed to access port 5000 on the Java app host. Note that this does not turn on authentication or encryption, but these can be added.
The following error messages assume the host config is set to 172.17.0.3 and the port set to 5000. Your host config and port settings might be different. The following sections show errors you might receive and their meanings:
Connection Refused π
java \
-Dcom.sun.management.jmxremote.port=5000 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.rmi.port=5000 \
...
This error indicates that the JMX connect port is not open on the specified host. Confirm, using netstat/ss or some other tool, that this port is indeed open on the configured host and is listening on an appropriate address. If the agent is running on a remote server, JMX might not be listening on localhost only.
RMI Connection Issues π
Creating MBean server connection failed: java.rmi.ConnectException: Connection refused to host: 172.17.0.3; nested exception is:
java.net.ConnectException: Connection timed out (Connection timed out)
This indicates that the JMX connect port was reached successfully, but the RMI port that it was directed to is being blocked, probably by a firewall. Make sure the com.sun.management.jmxremote.rmi.port
property in your Java app is set to the same port as the JMX connect port. There might be other variations of the error message that say Connection reset or Connection refused which indicate a similar cause.