Docs » Get started with the Splunk Distribution of the OpenTelemetry Collector » Collector components » Splunk HEC exporter

Splunk HEC exporter ๐Ÿ”—

The Splunk HTTP Event Collector (HEC) exporter allows the OpenTelemetry Collector to send traces, logs, and metrics to Splunk HEC endpoints. The supported pipeline types are traces, metrics, and logs. See Process your data with pipelines for more information.

The main purpose of the Splunk HEC exporter is to send logs and metrics to Splunk Cloud Platform or Splunk Enterprise. Log Observer Connect is now used to pull the Splunk Cloud Platform and Splunk Enterprise indexes into Splunk Observability Cloud. See Splunk Log Observer Connect for more information.

The exporter also sends AlwaysOn Profiling data to Splunk Observability Cloud. For more information, see Get data into Splunk APM AlwaysOn Profiling.

For information about the HEC receiver, see Splunk HEC receiver.

Caution

Splunk Log Observer is no longer available for new users. You can continue to use Log Observer if you already have an entitlement.

Get started ๐Ÿ”—

Note

This component is included in the default configuration of the Splunk Distribution of the OpenTelemetry Collector when deploying in host monitoring (agent) mode in the logs pipeline. See Collector deployment modes for more information.

For details about the default configuration, see Configure the Collector for Kubernetes with Helm, Collector for Linux default configuration, or Collector for Windows default configuration. You can customize your configuration any time as explained in this document.

Starting from version 0.81 of the Splunk Distribution of OpenTelemetry Collector, the default configuration includes an exporter for AlwaysOn Profiling data that is separate from the standard logs exporter. See Turn off logs or profiling data.

Follow these steps to configure and activate the component:

  1. Deploy the Splunk Distribution of OpenTelemetry Collector to your host or container platform:

  1. Configure the exporter as described in this doc.

  2. Restart the Collector.

Sample configuration ๐Ÿ”—

The following example shows a Splunk HEC exporter instance configured for a logs pipeline in the Collector configuration file:

exporters:
  # ...
  splunk_hec:
    token: "<hec-token>"
    endpoint: "<hec-endpoint>"
    # Source. See https://docs.splunk.com/Splexicon:Source
    source: "otel"
    # Source type. See https://docs.splunk.com/Splexicon:Sourcetype
    sourcetype: "otel"

# ...

Next, add the exporter to the services section of your configuration file:

service:
  # ...
  pipelines:
    logs:
      receivers: [fluentforward, otlp]
      processors:
      - memory_limiter
      - batch
      - resourcedetection
      exporters: [splunk_hec]

The Splunk HEC exporter requires a Splunk HEC token and endpoint. Obtaining a HEC token and choosing a HEC endpoint depends on the target. The following table shows endpoints and instructions for each back end. Use the source and sourcetype fields options when sending logs to Splunk Cloud Platform or Splunk Enterprise.

Back end

Endpoint

Tokens

Splunk Cloud Platform

See Send data to HTTP Event Collector on Splunk Cloud Platform

See Manage HTTP Event Collector (HEC) tokens in Splunk Cloud Platform

Splunk Enterprise

See Send data to HTTP Event Collector on Splunk Enterprise

See Create an Event Collector token on Splunk Enterprise

Splunk Observability Cloud

See Create and manage organization access tokens using Splunk Observability Cloud.

https://ingest.<realm>.signalfx.com/v1/log, where <realm> is the Splunk Observability Cloud realm, for example us0.

In the ingest endpoint URL, realm is the Splunk Observability Cloud realm, for example, us0. To find the realm name of your account, follow these steps:

  1. Open the navigation menu in Splunk Observability Cloud.

  2. Select Settings.

  3. Select your username.

The realm name appears in the Organizations section.

Note

To send Splunk HEC data through a proxy, configure proxy settings as environment variables. See Configure proxy settings for the Collector for more information.

Use cases ๐Ÿ”—

Send logs to Splunk Cloud Platform or Splunk Enterprise ๐Ÿ”—

If youโ€™re using the Collector for log collection and need to send data to Splunk Cloud Platform or Splunk Enterprise, configure the splunk_hec exporter to use your Splunk endpoint and token. For example:

exporters:
   splunk_hec:
      # Splunk HTTP Event Collector token.
      token: "00000000-0000-0000-0000-0000000000000"
      # URL to a Splunk instance to send data to.
      endpoint: "https://splunk:8088/services/collector"
      # Optional Splunk source: https://docs.splunk.com/Splexicon:Source
      source: "otel"
      # Optional Splunk source type: https://docs.splunk.com/Splexicon:Sourcetype
      sourcetype: "otel"
      # Splunk index, optional name of the Splunk index targeted.
      index: "metrics"
      # Maximum HTTP connections to use simultaneously when sending data. Defaults to 100.
      max_connections: 20
      # Whether to deactivate gzip compression over HTTP. Defaults to false.
      disable_compression: false
      # HTTP timeout when sending data. Defaults to 10s.
      timeout: 10s
      # Whether to skip checking the certificate of the HEC endpoint when sending data over HTTPS. Defaults to false.
      tls:
        insecure_skip_verify: true

You can split log data between Splunk Cloud Platform or Enterprise and Splunk Observability Cloud to preserve AlwaysOn Profiling data while sending logs to Splunk. See Check the OpenTelemetry Collector configuration for more information.

To split the log pipelines, configure two separate splunk_hec entries in the receiver and exporters sections of the Collector configuration file. Then, add both to the logs pipeline. For example:

receivers:
  # Default OTLP receiver--used by Splunk platform logs
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
   # OTLP receiver for AlwaysOn Profiling data
  otlp/profiling:
    protocols:
      grpc:
      # Make sure to configure your agents
      # to use the custom port for logs when
      # setting SPLUNK_PROFILER_LOGS_ENDPOINT
        endpoint: 0.0.0.0:4319

exporters:
  # Export logs to Splunk platform
  splunk_hec/platform:
    token: "<splunk_token>"
    endpoint: "https://splunk:8088/services/collector"
    source: "otel"
    sourcetype: "otel"
    index: "main"
    max_connections: 20
    disable_compression: false
    timeout: 10s
    tls:
      insecure_skip_verify: true
   # Export profiling data to Splunk Observability Cloud
  splunk_hec/profiling:
    token: "<splunk_o11y_token>"
    endpoint: "https://ingest.<realm>.signalfx.com/v1/log"
    source: "otel"
    sourcetype: "otel"
    log_data_enabled: false

processors:
  batch:
  memory_limiter:
    check_interval: 2s
    limit_mib: ${SPLUNK_MEMORY_LIMIT_MIB}

# Other settings

service:
  pipelines:
    # Traces and metrics pipelines
    # Logs pipeline for Splunk platform
    logs/platform:
      receivers: [fluentforward, otlp]
      processors:
      - memory_limiter
      - batch
      exporters: [splunk_hec/platform]
     # Logs pipeline for AlwaysOn Profiling
    logs/profiling:
      receivers: [otlp/profiling]
      processors:
      - memory_limiter
      - batch
      exporters: [splunk_hec/profiling]

Turn off logs or profiling data ๐Ÿ”—

Note

Starting from version 0.81 of the Splunk Distribution of OpenTelemetry Collector, logs and profiling pipelines are split. In that case, you can remove or comment them out according to your needs.

If you donโ€™t need AlwaysOn Profiling data for a specific host or container. set the profiling_data_enabled option to false in the splunk_hec exporter settings of the Collector configuration file. For example:

splunk_hec:
  token: "${SPLUNK_HEC_TOKEN}"
  endpoint: "${SPLUNK_HEC_URL}"
  source: "otel"
  sourcetype: "otel"
  profiling_data_enabled: false

To turn off log collection for Splunk Observability Cloud while preserving AlwaysOn Profiling data for APM, set the log_data_enabled option to false. See Exclude log data in the Collector for more information.

splunk_hec/profiling:
  token: "${SPLUNK_HEC_TOKEN}"
  endpoint: "${SPLUNK_HEC_URL}"
  source: "otel"
  sourcetype: "otel"
  log_data_enabled: false

If you need to turn off log data export to Splunk Observability Cloud, for example because youโ€™re using Log Observer Connect or because you donโ€™t have Log Observer in your organization, set log_data_enabled to false in the splunk_hec exporter of your Collector configuration file:

splunk_hec:
  token: "${SPLUNK_HEC_TOKEN}"
  endpoint: "${SPLUNK_HEC_URL}"
  source: "otel"
  sourcetype: "otel"
  log_data_enabled: false

Note

The log_data_enabled setting is available in the Splunk Distribution of OpenTelemetry Collector version 0.49.0 and higher.

To use a custom configuration for EC2, see Use a custom configuration. To use a custom configuration for Fargate, see Use a custom configuration.

If youโ€™ve deployed the Collector in Kubernetes using the Helm chart, change the following setting in the splunkObservability section of your custom chart or values.yaml file:

splunkObservability:
  # Other settings
  logsEnabled: false

Send metrics to Splunk Cloud Platform or Splunk Enterprise ๐Ÿ”—

You can use the Collector to send metrics to Splunk Cloud Platform or Splunk Enterprise.

For example, if youโ€™re scraping Prometheus metrics with a configuration such as:

pipelines:
  metrics:
      receivers: [prometheus]
      processors: [batch]
      exporters: [splunk_hec/metrics]

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'otel-collector'
          scrape_interval: 5s
          static_configs:
            - targets: ['<container_name>:<container_port>']

You need to configure the splunk_hec exporter as shown in the following example:

exporters:
    splunk_hec/metrics:
        # Splunk HTTP Event Collector token.
        token: "00000000-0000-0000-0000-0000000000000"
        # URL to a Splunk instance to send data to.
        endpoint: "https://splunk:8088/services/collector"
        # Optional Splunk source: https://docs.splunk.com/Splexicon:Source
        source: "app"
        # Optional Splunk source type: https://docs.splunk.com/Splexicon:Sourcetype
        sourcetype: "jvm_metrics"
        # Splunk index, optional name of the Splunk index targeted.
        index: "metrics"

Note that to be able to ingest metrics through Splunk HEC you need to declare your index as a metric index. To learn more about our metric index technology, see Get started with metrics in Splunk docs.

Settings ๐Ÿ”—

The following table shows the configuration options for the Splunk HEC exporter:

Troubleshooting ๐Ÿ”—

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways.

Available to Splunk Observability Cloud customers

Available to prospective customers and free trial users

  • Ask a question and get answers through community support at Splunk Answers .

  • Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide. To join, see Chat groups in the Get Started with Splunk Community manual.

To learn about even more support options, see Splunk Customer Success .