Docs » Get started with the Splunk Distribution of the OpenTelemetry Collector » Discover telemetry sources automatically » Automatic discovery and configuration for Kubernetes » Advanced customization for automatic discovery in Kubernetes

Advanced customization for automatic discovery in Kubernetes 🔗

Learn how to customize Splunk automatic discovery and configuration for advanced scenarios.

Through advanced customization, you can achieve the following tasks:

Note

For advanced customization options in Linux, see Advanced customization for automatic discovery in Linux.

Change instrumentation version 🔗

By default, the Splunk Distribution of OpenTelemetry Collector uses the latest version of the related instrumentation library. To change the version of your instrumentation library, follow these steps:

  1. Open your values.yaml file.

  2. In the instrumentation language section, change the tag value to match your desired version. The following example changes the Java instrumentation version to v1.27.0.

    clusterName: myCluster
splunkObservability:
  realm: <splunk-realm>
  accessToken: <splunk-access-token>
environment: prd
certmanager:
  enabled: true
operator:
  enabled: true
  instrumentation:
    spec:
      java:
        repository: ghcr.io/signalfx/splunk-otel-java/splunk-otel-java
        tag: v1.27.0

  3. Reinstall the Splunk OTel Collector Chart with the following command. Replace <CURRENT_VERSION> with the current version of your splunk-otel-collector-chart.

    helm upgrade splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector --version <CURRENT_VERSION> -f values.yaml

Note

If you don’t see the java field in your values.yaml file, then you have to add the field and the repository value before changing the version. The repository value is always ghcr.io/signalfx/splunk-otel-java/splunk-otel-java.

See the following pages for information about previous versions for each language:

Override default instrumentation settings 🔗

You can override default automatic discovery settings to use features for profiling.

Configure AlwaysOn Profiling 🔗

You can configure AlwaysOn Profiling in Kubernetes by editing the values.yaml file for the Helm Chart.

Follow these steps to activate Profiling for a language:

  1. Open the values.yaml file.

  2. In the operator.instrumentation.spec.<language>.env section, add the SPLUNK_PROFILER_ENABLED=true environment variable and the SPLUNK_PROFILER_CALL_STACK_INTERVAL environment variable.

    For example, the following values.yaml file configures AlwaysOn Profiling to sample call stacks from a 5000 millisecond interval:

    operator:
  enabled: true
instrumentation:
  spec:
    nodejs:
      env:
      # Activates AlwaysOn Profiling for Node.js
      - name: SPLUNK_PROFILER_ENABLED
        value: true
      # Samples call stacks from a 5000 millisecond interval.
      # If excluded, samples from a 10000 millisecond interval by default.
      - name: SPLUNK_PROFILER_CALL_STACK_INTERVAL
        value: 5000

  3. Reinstall the Splunk OTel Collector Chart with the following command. Replace <CURRENT_VERSION> with the current version of your splunk-otel-collector-chart.

    helm upgrade splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector --version <CURRENT_VERSION> -f values.yaml

Use automatic discovery with gateway mode 🔗

The Splunk OTel Collector Chart uses the agent mode by default. Activating gateway mode deploys an instance of the OpenTelemetry Collector in a separate container, and this instance collects data from the entire cluster.

To learn more about the gateway mode, see Data forwarding (gateway) mode.

Follow these steps to send data to a gateway endpoint:

  1. Open the values.yaml file.

  2. Set the operator.instrumentation.spec.exporter.endpoint value to the gateway endpoint. For example:

    clusterName: myCluster
splunkObservability:
  realm: <splunk-realm>
  accessToken: <splunk-access-token>
environment: prd
certmanager:
  enabled: true
operator:
    enabled: true
    instrumentation:
      spec:
        exporter:
          endpoint: <gateway-endpoint>

  3. Reinstall the Splunk OTel Collector Chart with the following command. Replace <CURRENT_VERSION> with the current version of your splunk-otel-collector-chart.

    helm upgrade splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector --version <CURRENT_VERSION> -f values.yaml

You can also automatically send data to a deployed gateway endpoint by setting agent.enabled: false and gateway.enabled: true in your values.yaml file. For example:

clusterName: myCluster
splunkObservability:
  realm: <splunk-realm>
  accessToken: <splunk-access-token>
environment: prd
certmanager:
  enabled: true
operator:
  enabled: true

agent:
  enabled: false
gateway:
  enabled: true

Using this configuration, automatic discovery automatically sends data to a running gateway endpoint.

Additional settings 🔗

There are many other settings you can customize in automatic discovery and configuration.

For a list of settings that you can change for each language, see the following resources:

Language

Resource

Java

Configure the Java agent for Splunk Observability Cloud

Node.js

Configure the Splunk Distribution of OTel JS for Splunk Observability Cloud

For a complete list of settings that you can modify in values.yaml for Kubernetes, see https://github.com/signalfx/splunk-otel-collector-chart/blob/main/helm-charts/splunk-otel-collector/values.yaml.