Get AlwaysOn Profiling data into Splunk APM 🔗
Prerequisites 🔗
To get data into Splunk AlwaysOn Profiling, you need the following:
Splunk APM enabled for your Observability Cloud organization.
The Splunk Distribution of OpenTelemetry Collector version 0.44.0 or higher. See Get started with the Splunk Distribution of the OpenTelemetry Collector.
If the version of your Splunk OTel Collector is lower than 0.44.0, see Check the OpenTelemetry Collector configuration.
Note
AlwaysOn Profiling is enabled for all host-based subscriptions. For TAPM-based subscriptions, check with your Splunk support representative.
Get profiling data in 🔗
Follow these instructions to get profiling data into Splunk APM using AlwaysOn Profiling:
Instrument your application or service 🔗
AlwaysOn Profiling requires APM tracing data to correlate stack traces to your application requests. To instrument your application for Splunk APM, follow the steps for the appropriate programming language:
Language |
Available instrumentation |
Documentation |
---|---|---|
Java |
Splunk Distribution of OpenTelemetry Java version 1.14.2 or higher |
Instrument a Java application for Splunk Observability Cloud, Java settings for AlwaysOn Profiling |
Node.js |
Splunk Distribution of OpenTelemetry JS version 2.0 or higher |
Instrument a Node application for Splunk Observability Cloud |
.NET |
SignalFx Instrumentation for .NET version 1.0.0 or higher |
Instrument a .NET application for Splunk Observability Cloud |
Note
See Data retention in Application Performance Monitoring (APM) for information on Profiling data retention.
Enable AlwaysOn Profiling 🔗
After you’ve instrumented your service for Observability Cloud and checked that APM data is getting into Splunk APM, enable AlwaysOn Profiling.
To enable AlwaysOn Profiling, follow the steps for the appropriate programming language:
To use CPU profiling, enable the
splunk.profiler.enabled
system property, or set theSPLUNK_PROFILER_ENABLED
environment variable totrue
.Enable Memory profiling by setting the
splunk.profiler.memory.enabled
system property or theSPLUNK_PROFILER_MEMORY_ENABLED
environment variable totrue
. To enable memory profiling, thesplunk.profiler.enabled
property must be set totrue
.Make sure that the
splunk.profiler.logs-endpoint
system property or theSPLUNK_PROFILER_LOGS_ENDPOINT
environment variable points tohttp://localhost:4317
.Port 9943 is the default port for the SignalFx receiver in the collector distribution. If you change this port in your Collector config, you need to pass the custom port to the JVM.
The following example shows how to enable the profiler using the system property:
java -javaagent:./splunk-otel-javaagent.jar \
-Dsplunk.profiler.enabled=true \
-Dsplunk.profiler.memory.enabled=true \
-Dotel.exporter.otlp.endpoint=http(s)://collector:4317 \
-Dsplunk.metrics.endpoint=http(s)://collector:9943
-jar <your_application>.jar
For more configuration options, including setting a separate endpoint for profiling data, see Java settings for AlwaysOn Profiling.
Note
Port 9943 is the default port for the SignalFx receiver in the collector distribution. If you change this port in your collector config, you need to pass the custom port to the JVM.
AlwaysOn Profiling requires Node 16 and higher.
Enable the profiler by setting the
SPLUNK_PROFILER_ENABLED
environment variable totrue
.Enable Memory profiling by setting the
SPLUNK_PROFILER_MEMORY_ENABLED
environment variable totrue
.Make sure that the
SPLUNK_PROFILER_LOGS_ENDPOINT
environment variable points tohttp://localhost:4317
or to the Splunk Distribution of OpenTelemetry Collector.
The following example shows how to enable the profiler from your application’s code:
start({
serviceName: '<service-name>',
endpoint: 'collectorhost:port',
profiling: { // Enables CPU profiling
memoryProfilingEnabled: true, // Enables Memory profiling
}
});
For more configuration options, including setting a separate endpoint for profiling data, see Node.js settings for AlwaysOn Profiling.
AlwaysOn Profiling requires NET Core 3.1 or .NET 5.0 and higher. Memory profiling requires .NET 5.0 and higher.
Enable the profiler by setting the
SIGNALFX_PROFILER_ENABLED
environment variable totrue
for your .NET process.Enable Memory profiling by setting the
SIGNALFX_PROFILER_MEMORY_ENABLED
environment variable totrue
.Make sure that the
SPLUNK_PROFILER_LOGS_ENDPOINT
environment variable points tohttp://localhost:4317
.Check that the
SIGNALFX_PROFILER_LOGS_ENDPOINT
environment variable points tohttp://localhost:4318/v1/logs
or to the Splunk Distribution of OpenTelemetry Collector.
For more configuration options, including setting a separate endpoint for profiling data, see .NET settings for AlwaysOn Profiling.
Check that Observability Cloud is receiving profiling data 🔗
After you set up and enable AlwaysOn Profiling, check that profiling data is coming in:
Log in to Splunk Observability Cloud.
In the left navigation menu, select
.In Splunk APM, select AlwaysOn Profiling.
Select a service, and switch from the CPU view to the Memory view.
If your service runs in multiple instances, select the instance that you’re interested in by selecting the host, container and process ID.
If you’ve enabled Memory profiling, explore memory metrics. See Memory profiling metrics.