Send data from Edge Processors to non-connected Splunk platform deployments using HEC

When sending data from an Edge Processor to a Splunk Enterprise deployment or a Splunk Cloud Platform deployment that is not connected to your tenant, you can choose to send that data using the HTTP Event Collector (HEC). HEC is a mechanism that allows HTTP clients and logging agents to send data to the Splunk platform over HTTP or HTTPS.

There are multiple HEC endpoints available, but the Edge Processor supports only the services/collector endpoint when sending data out through HEC. See services/collector in the Splunk Enterprise REST API Reference Manual for more information.

Start by adding a Splunk platform HEC destination in the Edge Processor service. You can configure the destination to send data to a specific Splunk platform instance, or to a load balancer or DNS that passes data to multiple instances. Splunk platform HEC destinations cannot send data to multiple instances directly.

Then, create a pipeline that uses that destination. When you apply that pipeline to your Edge Processor, the Edge Processor starts sending the data that it receives to your Splunk platform deployment.

The specific index that the data from an Edge Processor gets routed to is determined by a precedence order of configurations. For more information, see Index precedence order when using HEC.

You can also send data using the Splunk-to-Splunk (S2S) protocol instead of HEC, or send data to the Splunk Cloud Platform deployment that is connected to your tenant without needing to add any destinations. For more information, see Sending data from Edge Processors to Splunk Cloud Platform or Splunk Enterprise.

Precedence order of HEC tokens and metadata field values

When configuring a Splunk platform HEC destination, you must specify a default HEC token. This default token is used only if the data is not already associated with a HEC token. For example, if the Edge Processor received an event through HEC, and the Authorization header in the HTTP request that transmitted that event includes a HEC token, then the token in the header is used when you send this event from your Edge Processor to the Splunk platform.

Additionally, you can specify default values for some of the metadata fields in the events.

Source field

When you send data out from an Edge Processor using a Splunk platform HEC destination, the value of the source field is determined based on the following precedence order:

1. The source value that is already specified in the event before the Edge Processor receives it.
2. The Default source setting specified in the Splunk platform HEC destination.
3. The Source name override setting specified in the HEC token being used.

Sourcetype field

Edge Processor pipelines only accept events that have a value in the sourcetype field. These sourcetype values remain unchanged when you send processed data from a pipeline to a Splunk platform HEC destination.

However, if you select a Splunk platform HEC destination as the default destination for an Edge Processor, then unprocessed events without sourcetype values might be sent through that destination. In this case, the value of the sourcetype field is determined based on the following precedence order:

1. The Default source type setting specified in the Splunk platform HEC destination.
2. The Source type setting specified in the HEC token being used.
3. The Default Source Type setting specified in the HEC shared settings of a Splunk Enterprise deployment. This setting is applicable only when you are sending data to Splunk Enterprise.

Index field

The index value is determined based on an extensive precedence order of configurations. See Index precedence order when using HEC for more information.

TLS and mTLS support

When sending data from an Edge Processor to a Splunk indexer using HEC, in most cases you can choose to secure communications using TLS or mutually authenticated TLS (mTLS).

Using TLS when sending data from Edge Processors to indexes through HEC

Splunk Enterprise and Splunk Cloud Platform indexers both support TLS. Splunk Cloud Platform indexers always require TLS.

When TLS is used, the Edge Processor requires the indexer to prove its identity using a valid set of TLS certificates. If the indexer cannot provide these certificates, then the Edge Processor does not connect to the indexer and does not send any data to it.

To use TLS, when configuring your Splunk platform HEC destination, make sure that the HEC URI value starts with https instead of http.

Using mTLS when sending data from Edge Processors to indexes through HEC

Splunk Cloud Platform indexers do not support mTLS for HEC connections. Only Splunk Enterprise indexes support mTLS.

When mTLS is used, both the Edge Processor and the indexer must prove their identities using valid TLS certificates. If either system cannot provide these certificates, then the Edge Processor does not connect to the indexer and does not send any data to it.

To use mTLS, when configuring your Splunk platform HEC destination, you must turn on the Authenticate identity using TLS certificates setting and then upload a client certificate, private key, and CA certificates. See the rest of this page for more information.

Prerequisites

Before you can add a destination that sends data to the Splunk platform using HEC, you must do the following:

• In the Splunk platform deployment, turn on the HTTP Event Collector.
• Turn on the HEC token that you want to use, and make sure that the token configuration meets these requirements:
  • The Enable indexer acknowledgement setting is turned off.
  • The token allows data to be sent to all indexes. In the token configuration settings in Splunk Web, make sure that the Selected Indexes pane of the Select Allowed Indexes control is empty.

  If you try to send data from your Edge Processor using a HEC token that doesn't meet these requirements, data loss can occur.

• If you're planning to send data to multiple Splunk platform instances, such as multiple indexers, then you must configure a load balancer or DNS to pass the data from the Edge Processor to those instances.
• Make note of one of the following values, depending on how you plan to send your data:
  • If you're sending data to a specific Splunk platform instance, then make note of the HEC URI for that instance. For information about HEC URI formats, see the following sections in the Splunk Cloud Platform Getting Data In manual:
    • Send data to HTTP Event Collector on Splunk Cloud Platform
    • Send data to HTTP Event Collector on Splunk Enterprise
  • If you're using a load balancer or DNS to send data to multiple Splunk platform instances, then make note of the URL of the load balancer or DNS.
• If you're sending data to a Splunk Enterprise indexer that has the enableSSL property set to 1 in the inputs.conf file, that means the indexer uses mTLS for HEC connections and requires connecting clients to authenticate themselves using TLS certificates. In this case, you must obtain certificates for proving the Edge Processor's identity. See the Obtaining TLS certificates section in this topic for more information.

Obtaining TLS certificates

If you're sending data to a Splunk Enterprise indexer that uses mTLS, then you need to have TLS certificates that the Edge Processor can use to prove its identity to the indexer.

Obtain the following certificates, contained in separate Privacy Enhanced Mail (PEM) files:

• A client certificate.
• The private key associated with that client certificate. This private key must be decrypted.
• The CA certificates used to verify the indexer.

If you don't have these PEM files, ask your Splunk Enterprise administrator for assistance. See the Secure Splunk platform communications with Transport Layer Security certificates chapter of the Securing Splunk Enterprise manual for more information.

Add a Splunk platform HEC destination

1. In the Edge Processor service, select Destinations.
2. On the Destinations page, select New destination, then Splunk platform using HEC.
3. Provide a name and description for your destination.

   Field	Description
   Name	A unique name for your destination
   Description	(Optional) A description of your destination

4. In the HEC URI field, enter one of these values:
   • The HEC URI of the Splunk platform instance that you want to send data to. This URI must point to the services/collector HEC endpoint.
   • The URL of a load balancer or DNS that you're using to send data to multiple Splunk platform instances.

   
   The HEC URI or URL must start with https instead of http if any of the following conditions are true:

   • You want the Edge Processor to verify the identity of the Splunk platform instance, load balancer, or DNS using TLS.
   • You're sending data to a Splunk Cloud Platform indexer. Splunk Cloud Platform indexers always require TLS.
   • You're sending data to a Splunk Enterprise indexer that uses mTLS and requires the Edge Processor to prove its identity using TLS certificates.
5. In the Default HEC token field, enter the value of a HEC token from your Splunk platform deployment. This HEC token is used only when the Edge Processor is sending out data that is not already associated with a HEC token.
6. (Optional) Provide default values for the metadata fields in the events that are sent through this destination. These values are used only if the events do not already contain source, sourcetype, or index values.

   Field	Description
   Default source	The name of the source from which the event originates.
   Default source type	A value that identifies the data structure of the event. Edge Processor pipelines only accept events that are already associated with a source type, so the Default source type value does not affect events in pipelines. This value is used only when you select a Splunk platform HEC destination as the default destination for an Edge Processor, and events without source types are routed through the destination.
   Default index	The name of the Splunk index that the Edge Processor sends the event to.

7. If you're sending data to a Splunk instance that doesn't use mTLS, then skip this step. If you're sending data to a Splunk Enterprise indexer that uses mTLS, then do the following:
   1. Select Authenticate identity using TLS certificates.
   2. Upload the appropriate private key and certificates in these fields:

      Field	Description
      Client private key	A PEM file containing the decrypted private key associated with your client certificate
      Client certificate	A PEM file containing a client certificate
      CA certificates	The CA certificates used to verify the indexer

8. To finish adding the destination, select Add.

You now have a destination that you can use to send data from an Edge Processor to one or more Splunk platform instances using HEC.

To start sending data, create a pipeline that uses the destination you just added and then apply that pipeline to your Edge Processor. For more information, see Create pipelines for Edge Processors and Apply pipelines to Edge Processors.

See also

For information about configuring HEC in the Splunk platform, see Set up and use HTTP Event Collector in Splunk Web in the Splunk Cloud Platform Getting Data In manual.