Sending data from Edge Processors to Splunk Cloud Platform or Splunk Enterprise
You can send data from Edge Processors to Splunk Enterprise or Splunk Cloud Platform. The steps that you need to take in order to send data to a Splunk platform deployment varies depending on these factors:
- Are you sending data to the Splunk Cloud Platform deployment that is connected to the Edge Processor service?
- Do you want to send this data using the Splunk-to-Splunk (S2S) protocol or the HTTP Event Collector (HEC)?
During the first-time setup process for the Edge Processor solution, the Edge Processor service is connected to a Splunk Cloud Platform deployment. Due to this connection, the indexers associated with this deployment are already available as data destinations for Edge Processor pipelines. You can create a pipeline to send data to the connected Splunk Cloud Platform deployment using the S2S protocol. For more information, see Send data from Edge Processors to the Splunk Cloud Platform deployment connected to your tenant.
Before you can send data to a non-connected Splunk Cloud Platform or Splunk Enterprise deployment, you must add the indexers from those deployments as destinations in the Edge Processor service. When sending data to a non-connected Splunk platform deployment, you can choose to use the S2S protocol or HEC:
- S2S is the proprietary, TCP-based data transmission protocol used between Splunk software. The S2S protocol typically sends data faster and more efficiently than HEC, and does not require any additional configurations on the Splunk platform deployment. To send data using S2S, add and use a Splunk platform S2S destination. See Send data from Edge Processors to non-connected Splunk platform deployments using S2S for more information.
- HEC is a mechanism that allows HTTP clients and logging agents to send data to the Splunk platform over HTTP or HTTPS. If your Splunk platform deployment has HEC turned on and valid HEC tokens configured, then you can choose to send data using HEC. To do this, add and use a Splunk platform HEC destination. See Send data from Edge Processors to non-connected Splunk platform deployments using HEC for more information.
The protocol that you use to send the data affects how that data gets routed to an index. See the rest of this topic for details.
How does an Edge Processor know which index to send data to?
The specific index that the data from an Edge Processor gets routed to is determined by a precedence order of configurations. See the following tables for details:
Edge Processors use the S2S protocol when sending data to the Splunk Cloud Platform deployment that's connected to the tenant.
Index precedence order when using S2S
When you use the S2S protocol to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:
Configuration | Description |
---|---|
Data routing configurations in the Splunk platform deployment | If the deployment is configured to route events to different indexes based on field values, then the Edge Processor sends data to the index determined by these routing configurations.
|
The SPL2 statement of the pipeline | If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index.
$pipeline = | from $source | eval index="AppLogEvents" | into $destination;
|
The metadata in the event payload | If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index.
|
None of the previously described configurations specify an index | The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information. |
If the destination index determined by this precedence order does not exist in the Splunk platform deployment, then one of the following outcomes occur:
- If the
lastChanceIndex
property is configured in the Splunk platform deployment, then the data goes to the index specified by that property. - If the
lastChanceIndex
property is not configured, then the data is dropped.
For more information about the lastChanceIndex
property, see indexes.conf in the Splunk Enterprise Admin Manual.
Index precedence order when using HEC
When you use HEC to send data from an Edge Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:
Configuration | Description |
---|---|
The SPL2 statement of the pipeline | If the pipeline contains an eval command that sets the index field to a specific value, then the Edge Processor sends data to the specified index.
$pipeline = | from $source | eval index="AppLogEvents" | into $destination;
|
The metadata in the event payload | If the event contains metadata that specifies an index, then the Edge Processor sends the event to that index.
|
The Default index configuration in a Splunk platform HEC destination | If the pipeline uses a Splunk platform HEC destination, and the Default index setting in the destination specifies an index name, then the Edge Processor sends data to that index. |
The Default Index configuration in the HEC token | If the pipeline uses a Splunk platform HEC destination, and the Default Index setting in the HEC token specifies an index name, then the Edge Processor sends data to that index. |
The Default Index configuration in the HEC shared settings of a Splunk Enterprise deployment | If you're sending data to Splunk Enterprise using a Splunk platform HEC destination, and the Default Index setting in the HEC shared settings of the Splunk Enterprise deployment specifies an index name, then the Edge Processor sends data to that index. |
None of the previously described configurations specify an index | The Edge Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information. |
If the destination index determined by this precedence order does not exist in the Splunk platform deployment, then one of the following outcomes occur:
- If the
lastChanceIndex
property is configured in the Splunk platform deployment, then the data goes to the index specified by that property. - If the
lastChanceIndex
property is not configured, then the data is dropped.
For more information about the lastChanceIndex
property, see indexes.conf in the Splunk Enterprise Admin Manual.
Add or manage destinations | Send data from Edge Processors to the Splunk Cloud Platform deployment connected to your tenant |
This documentation applies to the following versions of Splunk Cloud Platform™: 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release)
Feedback submitted, thanks!