Splunk Cloud Platform

Use Edge Processors

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Sending data from Edge Processors to Splunk Cloud Platform or Splunk Enterprise

You can send data from Edge Processors to Splunk Enterprise or Splunk Cloud Platform. The steps that you need to take in order to send data to a Splunk platform deployment varies depending on these factors:

  • Are you sending data to the Splunk Cloud Platform deployment that is connected to the Edge Processor service?
  • Do you want to send this data using the Splunk-to-Splunk (S2S) protocol or the HTTP Event Collector (HEC)?

During the first-time setup process for the Edge Processor solution, the Edge Processor service is connected to a Splunk Cloud Platform deployment. Due to this connection, the indexers associated with this deployment are already available as data destinations for Edge Processor pipelines. You can create a pipeline to send data to the connected Splunk Cloud Platform deployment using the S2S protocol. For more information, see Send data from Edge Processors to the Splunk Cloud Platform deployment connected to your tenant.

Before you can send data to a non-connected Splunk Cloud Platform or Splunk Enterprise deployment, you must add the indexers from those deployments as destinations in the Edge Processor service. When sending data to a non-connected Splunk platform deployment, you can choose to use the S2S protocol or HEC:

  • S2S is the proprietary, TCP-based data transmission protocol used between Splunk software. The S2S protocol typically sends data faster and more efficiently than HEC, and does not require any additional configurations on the Splunk platform deployment. To send data using S2S, add and use a Splunk platform S2S destination. See Send data from Edge Processors to non-connected Splunk platform deployments using S2S for more information.
  • HEC is a mechanism that allows HTTP clients and logging agents to send data to the Splunk platform over HTTP or HTTPS. If your Splunk platform deployment has HEC turned on and valid HEC tokens configured, then you can choose to send data using HEC. To do this, add and use a Splunk platform HEC destination. See Send data from Edge Processors to non-connected Splunk platform deployments using HEC for more information.

The protocol that you use to send the data affects how that data gets routed to an index. See the rest of this topic for details.

How does an Edge Processor know which index to send data to?

The specific index that the data from an Edge Processor gets routed to is determined by a precedence order of configurations. See the following tables for details:

Edge Processors use the S2S protocol when sending data to the Splunk Cloud Platform deployment that's connected to the tenant.

Index precedence order when using S2S

When you use the S2S protocol to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration Description
Data routing configurations in the Splunk platform deployment If the deployment is configured to route events to different indexes based on field values, then the Edge Processor sends data to the index determined by these routing configurations.


For example, if the props.conf file specifies a transforms.conf stanza, and that stanza uses the REGEX and DEST_KEY properties to route data to different indexes based on extracted field values, then data from the Edge Processor is routed according to these settings.

The SPL2 statement of the pipeline If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index.


For example, if you apply the following pipeline, then the Edge Processor sends data to an index called AppLogEvents:
$pipeline = | from $source | eval index="AppLogEvents" | into $destination;


You can also add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See Create pipelines for Edge Processors for more information.

The metadata in the event payload If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index.


The index in the event metadata can be set through various methods as the event travels from the original data source to the Edge Processor. For example:

  • When you use a Splunk forwarder to send the event to the Edge Processor, the index value in the inputs.conf file specifies the index in the event metadata.
  • When you use HEC to send the event to the Edge Processor, the index parameter in the HTTP request specifies the index in the event metadata.
None of the previously described configurations specify an index The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.

Index precedence order when using HEC

When you use HEC to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration Description
The SPL2 statement of the pipeline If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index.


For example, if you apply the following pipeline, then the Edge Processor sends data to an index called AppLogEvents:
$pipeline = | from $source | eval index="AppLogEvents" | into $destination;


You can also add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See Create pipelines for Edge Processors for more information.

The metadata in the event payload If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index.


The index in the event metadata can be set through various methods as the event travels from the original data source to the Edge Processor. For example:

  • When you use a Splunk forwarder to send the event to the Edge Processor, the index value in the inputs.conf file specifies the index in the event metadata.
  • When you use HEC to send the event to the Edge Processor, the index parameter in the HTTP request specifies the index in the event metadata.
The Default index configuration in a Splunk platform HEC destination If the pipeline uses a Splunk platform HEC destination, and the Default index setting in the destination specifies an index name, then the Edge Processor sends data to that index.
The Default Index configuration in the HEC token If the pipeline uses a Splunk platform HEC destination, and the Default Index setting in the HEC token specifies an index name, then the Edge Processor sends data to that index.
The Default Index configuration in the HEC global settings of a Splunk Enterprise deployment If you're sending data to Splunk Enterprise using a Splunk platform HEC destination, and the Default Index setting in the HEC global settings of the Splunk Enterprise deployment specifies an index name, then the Edge Processor sends data to that index.
None of the previously described configurations specify an index The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.
Last modified on 16 November, 2023
PREVIOUS
Add or manage destinations
  NEXT
Send data from Edge Processors to the Splunk Cloud Platform deployment connected to your tenant

This documentation applies to the following versions of Splunk Cloud Platform: 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308 (latest FedRAMP release), 9.1.2312


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters