Sending data from Edge Processors to Splunk Cloud Platform or Splunk Enterprise

You can send data from Edge Processors to Splunk Enterprise or Splunk Cloud Platform. The steps that you need to take in order to send data to a Splunk platform deployment varies depending on these factors:

• Are you sending data to the Splunk Cloud Platform deployment that is connected to the Edge Processor service?
• Do you want to send this data using the Splunk-to-Splunk (S2S) protocol or the HTTP Event Collector (HEC)?

During the first-time setup process for the Edge Processor solution, the Edge Processor service is connected to a Splunk Cloud Platform deployment. Due to this connection, the indexers associated with this deployment are already available as data destinations for Edge Processor pipelines. You can create a pipeline to send data to the connected Splunk Cloud Platform deployment using the S2S protocol. For more information, see Send data from Edge Processors to the Splunk Cloud Platform deployment connected to your tenant.

Before you can send data to a non-connected Splunk Cloud Platform or Splunk Enterprise deployment, you must add the indexers from those deployments as destinations in the Edge Processor service. When sending data to a non-connected Splunk platform deployment, you can choose to use the S2S protocol or HEC:

• S2S is the proprietary, TCP-based data transmission protocol used between Splunk software. The S2S protocol typically sends data faster and more efficiently than HEC, and does not require any additional configurations on the Splunk platform deployment. To send data using S2S, add and use a Splunk platform S2S destination. See Send data from Edge Processors to non-connected Splunk platform deployments using S2S for more information.
• HEC is a mechanism that allows HTTP clients and logging agents to send data to the Splunk platform over HTTP or HTTPS. If your Splunk platform deployment has HEC turned on and valid HEC tokens configured, then you can choose to send data using HEC. To do this, add and use a Splunk platform HEC destination. See Send data from Edge Processors to non-connected Splunk platform deployments using HEC for more information.

The protocol that you use to send the data affects how that data gets routed to an index. See the rest of this topic for details.

How does an Edge Processor know which index to send data to?

The specific index that the data from an Edge Processor gets routed to is determined by a precedence order of configurations. See the following tables for details:

• Index precedence order when using S2S
• Index precedence order when using HEC

Edge Processors use the S2S protocol when sending data to the Splunk Cloud Platform deployment that's connected to the tenant.

Index precedence order when using S2S

When you use the S2S protocol to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration	Description
Data routing configurations in the Splunk platform deployment	If the deployment is configured to route events to different indexes based on field values, then the Edge Processor sends data to the index determined by these routing configurations. For example, if the props.conf file specifies a transforms.conf stanza, and that stanza uses the REGEX and DEST_KEY properties to route data to different indexes based on extracted field values, then data from the Edge Processor is routed according to these settings.
The SPL2 statement of the pipeline	If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index. For example, if you apply the following pipeline, then the Edge Processor sends data to an index called AppLogEvents: $pipeline = | from $source | eval index="AppLogEvents" | into $destination; You can add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See Create pipelines for Edge Processors for more information.
The metadata in the event payload	If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index. The index in the event metadata can be set through various methods as the event travels from the original data source to the Edge Processor. For example: When you use a Splunk forwarder to send the event to the Edge Processor, the index value in the inputs.conf file specifies the index in the event metadata. When you use HEC to send the event to the Edge Processor, the index parameter in the HTTP request specifies the index in the event metadata.
None of the previously described configurations specify an index	The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.

If the destination index determined by this precedence order does not exist in the Splunk platform deployment, then one of the following outcomes occur:

• If the lastChanceIndex property is configured in the Splunk platform deployment, then the data goes to the index specified by that property.
• If the lastChanceIndex property is not configured, then the data is dropped.

For more information about the lastChanceIndex property, see indexes.conf in the Splunk Enterprise Admin Manual.

Index precedence order when using HEC

When you use HEC to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration	Description
The SPL2 statement of the pipeline	If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index. For example, if you apply the following pipeline, then the Edge Processor sends data to an index called AppLogEvents: $pipeline = | from $source | eval index="AppLogEvents" | into $destination; You can add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See Create pipelines for Edge Processors for more information.
The metadata in the event payload	If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index. The index in the event metadata can be set through various methods as the event travels from the original data source to the Edge Processor. For example: When you use a Splunk forwarder to send the event to the Edge Processor, the index value in the inputs.conf file specifies the index in the event metadata. When you use HEC to send the event to the Edge Processor, the index parameter in the HTTP request specifies the index in the event metadata.
The Default index configuration in a Splunk platform HEC destination	If the pipeline uses a Splunk platform HEC destination, and the Default index setting in the destination specifies an index name, then the Edge Processor sends data to that index.
The Default Index configuration in the HEC token	If the pipeline uses a Splunk platform HEC destination, and the Default Index setting in the HEC token specifies an index name, then the Edge Processor sends data to that index.
The Default Index configuration in the HEC shared settings of a Splunk Enterprise deployment	If you're sending data to Splunk Enterprise using a Splunk platform HEC destination, and the Default Index setting in the HEC shared settings of the Splunk Enterprise deployment specifies an index name, then the Edge Processor sends data to that index.
None of the previously described configurations specify an index	The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.

If the destination index determined by this precedence order does not exist in the Splunk platform deployment, then one of the following outcomes occur:

• If the lastChanceIndex property is configured in the Splunk platform deployment, then the data goes to the index specified by that property.
• If the lastChanceIndex property is not configured, then the data is dropped.

For more information about the lastChanceIndex property, see indexes.conf in the Splunk Enterprise Admin Manual.