Docs ยป Supported integrations in Splunk Observability Cloud ยป Configure application receivers with Prometheus metrics ยป Prometheus Go

Prometheus Go ๐Ÿ”—

The Splunk Distribution of OpenTelemetry Collector uses the Smart Agent receiver with the prometheus-go monitor type to wrap the Prometheus Exporter to scrape Prometheus Go collector and Prometheus process collector metrics for Splunk Observability Cloud.

This integration is available on Linux and Windows.

Benefits ๐Ÿ”—

After you configure the integration, you can access these features:

Installation ๐Ÿ”—

Follow these steps to deploy this integration:

  1. Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:

  2. Configure the integration, as described in the Configuration section.

  3. Restart the Splunk Distribution of the OpenTelemetry Collector.

Configuration ๐Ÿ”—

To use this integration of a Smart Agent monitor with the Collector:

  1. Include the Smart Agent receiver in your configuration file.

  2. Add the monitor type to the Collector configuration, both in the receiver and pipelines sections.

Example ๐Ÿ”—

To activate this integration, add the following to your Collector configuration:

receivers:
  smartagent/prometheus-go:
    type: prometheus-go
    host: localhost
    port: 2112
    ... # Additional config

Next, add the monitor to the service.pipelines.metrics.receivers section of your configuration file:

service:
  pipelines:
    metrics:
      receivers: [smartagent/prometheus-go]

Configuration settings ๐Ÿ”—

The following table shows the configuration options for the prometheus-go monitor:

Option

Required

Type

Description

httpTimeout

no

int64

HTTP timeout duration for both reads and writes. Must be a

duration string accepted by https://golang.org/pkg/time/#ParseDuration. Default value is 10s.

username

no

string

User name to use on each request.

password

no

string

Password to use on each request.

useHTTPS

no

bool

If true, the agent connects to the server using HTTPS instead of

plain HTTP. Default value is false.

httpHeaders

no

map of strings

A map of HTTP header names to values. Comma-separated multiple

values for the same message-header are supported.

skipVerify

no

bool

If both useHTTPS and skipVerify are true, the TLS

certificate of the exporter is not verified. Default value is false.

caCertPath

no

string

Path to the CA certificate that has signed the TLS certificate,

unnecessary if skipVerify is set to false.

clientCertPath

no

string

Path to the client TLS certificate to use for TLS required

connections.

clientKeyPath

no

string

Path to the client TLS key to use for TLS required connections.

host

yes

string

Host of the exporter.

port

yes

integer

Port of the exporter.

useServiceAccount

no

bool

Use pod service account to authenticate. Default value is

false.

metricPath

no

string

Path to the metrics endpoint on the exporter server. The default

value is /metrics.

sendAllMetrics

no

bool

Send all the metrics that come out of the Prometheus exporter

without any filtering. This option has no effect when using the Prometheus exporter monitor directly, since there is no built-in filtering. Default value is false.

Metrics ๐Ÿ”—

The following metrics are available for this integration.

NameDescriptionType
go_gc_duration_seconds

A summary of the GC invocation durations

cumulative
go_gc_duration_seconds_bucket

A summary of the GC invocation durations

cumulative
go_gc_duration_seconds_count

A summary of the GC invocation durations

cumulative
go_goroutines

Number of goroutines that currently exist

gauge
go_info

Information about the Go environment

gauge
go_memstats_alloc_bytes

Number of bytes allocated and still in use

gauge
go_memstats_alloc_bytes_total

Total number of bytes allocated, even if freed

cumulative
go_memstats_buck_hash_sys_bytes

Number of bytes used by the profiling bucket hash table

gauge
go_memstats_frees_total

Total number of frees

cumulative
go_memstats_gc_cpu_fraction

The fraction of this program's available CPU time used by the GC since the program started

gauge
go_memstats_gc_sys_bytes

Number of bytes used for garbage collection system metadata

gauge
go_memstats_heap_alloc_bytes

Number of heap bytes allocated and still in use

gauge
go_memstats_heap_idle_bytes

Number of heap bytes waiting to be used

gauge
go_memstats_heap_inuse_bytes

Number of heap bytes that are in use

gauge
go_memstats_heap_objects

Number of allocated objects

gauge
go_memstats_heap_released_bytes

Number of heap bytes released to OS

gauge
go_memstats_heap_sys_bytes

Number of heap bytes obtained from system

gauge
go_memstats_last_gc_time_seconds

Number of seconds since 1970 of last garbage collection

gauge
go_memstats_lookups_total

Total number of pointer lookups

cumulative
go_memstats_mallocs_total

Total number of mallocs

cumulative
go_memstats_mcache_inuse_bytes

Number of bytes in use by mcache structures

gauge
go_memstats_mcache_sys_bytes

Number of bytes used for mcache structures obtained from system

gauge
go_memstats_mspan_inuse_bytes

Number of bytes in use by mspan structures

gauge
go_memstats_mspan_sys_bytes

Number of bytes used for mspan structures obtained from system

gauge
go_memstats_next_gc_bytes

Number of heap bytes when next garbage collection will take place

gauge
go_memstats_other_sys_bytes

Number of bytes used for other system allocations

gauge
go_memstats_stack_inuse_bytes

Number of bytes in use by the stack allocator

gauge
go_memstats_stack_sys_bytes

Number of bytes obtained from system for stack allocator

gauge
go_memstats_sys_bytes

Number of bytes obtained from system

gauge
go_threads

Number of OS threads created

gauge
process_cpu_seconds_total

Total user and system CPU time spent in seconds

cumulative
process_max_fds

Maximum number of open file descriptors

gauge
process_open_fds

Number of open file descriptors

gauge
process_resident_memory_bytes

Resident memory size in bytes

gauge
process_start_time_seconds

Start time of the process since unix epoch in seconds

gauge
process_virtual_memory_bytes

Virtual memory size in bytes

gauge
process_virtual_memory_max_bytes

Maximum amount of virtual memory available in bytes

gauge

Notes ๐Ÿ”—

  • To learn more about the available in Splunk Observability Cloud see Metric types

  • In host-based subscription plans, default metrics are those metrics included in host-based subscriptions in Splunk Observability Cloud, such as host, container, or bundled metrics. Custom metrics are not provided by default and might be subject to charges. See Metric categories for more information.

  • In MTS-based subscription plans, all metrics are custom.

  • To add additional metrics, see how to configure extraMetrics in Add additional metrics

Non-default metrics (version 4.7.0+) ๐Ÿ”—

To emit metrics that are not default, you can add those metrics in the generic receiver-level extraMetrics config option. You donโ€™t need to add to extraMetrics any metric derived from configuration options that donโ€™t appear in the list of metrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring the receiver in a running agent instance.

Troubleshooting ๐Ÿ”—

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways.

Available to Splunk Observability Cloud customers

Available to prospective customers and free trial users

  • Ask a question and get answers through community support at Splunk Answers .

  • Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide. To join, see Chat groups in the Get Started with Splunk Community manual.

This page was last updated on Dec 09, 2024.