Ingest Processor is currently released as a preview only and is not officially supported. See Splunk General Terms for more information. For any questions on this preview, please reach out to ingestprocessor@splunk.com.

Sending data from Ingest Processor to Splunk Cloud Platform or Splunk Enterprise

You can send data from Ingest Processor to Splunk Enterprise or Splunk Cloud Platform. The steps that you need to take in order to send data to a Splunk platform deployment varies depending on these factors:

• Are you sending data to the Splunk Cloud Platform deployment that is connected to the Ingest Processor service?
• Do you want to send this data using the Splunk-to-Splunk (S2S) protocol or the HTTP Event Collector (HEC)?

During the first-time setup process for the Ingest Processor solution, the Ingest Processor solution is connected to a Splunk Cloud Platform deployment. Due to this connection, the indexes and indexers associated with this deployment are already available as data destinations for Ingest Processor pipelines. You can create a pipeline to send data to the connected Splunk Cloud Platform deployment using the S2S protocol. For more information, see Send data from Ingest Processor to the Splunk Cloud Platform deployment connected to your tenant.

Before you can send data to a non-connected Splunk Cloud Platform or Splunk Enterprise deployment, you must add the indexers from those deployments as destinations in the Ingest Processor service. When sending data to a non-connected Splunk platform deployment, you can choose to use the S2S protocol or HEC:

• S2S is the proprietary, TCP-based data transmission protocol used between Splunk software. The S2S protocol typically sends data faster and more efficiently than HEC, and does not require any additional configurations on the Splunk platform deployment. To send data using S2S, add and use a Splunk platform S2S destination. See Send data from Ingest Processor to non-connected Splunk platform deployments using S2S for more information.
• HEC is a mechanism that allows HTTP clients and logging agents to send data to the Splunk platform over HTTP or HTTPS. If your Splunk platform deployment has HEC turned on and valid HEC tokens configured, then you can choose to send data using HEC. To do this, add and use a Splunk platform HEC destination. See Send data from Ingest Processor to non-connected Splunk platform deployments using HEC for more information.

The protocol that you use to send the data affects how that data gets routed to an index. See the rest of this topic for details.

How does Ingest Processor know which index to send data to?

The specific index that the data from Ingest Processor gets routed to is determined by a precedence order of configurations. See the following tables for details:

• Index precedence order when using S2S
• Index precedence order when using HEC

Ingest Processor use the S2S protocol when sending data to the Splunk Cloud Platform deployment that's connected to the tenant.

Index precedence order when using S2S

When you use the S2S protocol to send data from Ingest Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration	Description
Data routing configurations in the Splunk platform deployment	If the deployment is configured to route events to different indexes based on field values, then the Ingest Processor solution sends data to the index determined by these routing configurations. For example, if the props.conf file specifies a transforms.conf stanza, and that stanza uses the REGEX and DEST_KEY properties to route data to different indexes based on extracted field values, then data from Ingest Processor is routed according to these settings.
The SPL2 statement of the pipeline	If the pipeline contains an eval command that sets the index field to a specific value, then Ingest Processor sends data to the specified index. For example, if you apply the following pipeline, then the Ingest Processor solution sends data to an index called AppLogEvents: $pipeline = | from $source | eval index="AppLogEvents" | into $destination; You can also add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See the Create pipelines for Ingest Processor topic in this manual for more information.
The metadata in the event payload	If the event contains metadata that specifies an index, then Ingest Processor sends the event to that index. The index in the event metadata can be set through various methods as the event travels from the original data source to Ingest Processor. For example: When you use a Splunk forwarder to send the event to Ingest Processor, the index value in the inputs.conf file specifies the index in the event metadata. When you use HEC to send the event to Ingest Processor, the index parameter in the HTTP request specifies the index in the event metadata.
None of the previously described configurations specify an index	Ingest Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.

Index precedence order when using HEC

When you use HEC to send data from Ingest Processor to the Splunk platform, the destination index is determined by the following precedence order of configurations:

Configuration	Description
The SPL2 statement of the pipeline	If the pipeline contains an eval command that sets the index field to a specific value, then Ingest Processor sends data to the specified index. For example, if you apply the following pipeline, then Ingest Processor sends data to an index called AppLogEvents: $pipeline = | from $source | eval index="AppLogEvents" | into $destination; You can also add this command by specifying a target index during pipeline creation or by selecting the Target index action when editing a pipeline. See the Create pipelines for Ingest Processor topic in this manual for more information.
The metadata in the event payload	If the event contains metadata that specifies an index, then Ingest Processor sends the event to that index. The index in the event metadata can be set through various methods as the event travels from the original data source to Ingest Processor. For example: When you use a Splunk forwarder to send the event to Ingest Processor, the index value in the inputs.conf file specifies the index in the event metadata. When you use HEC to send the event to Ingest Processor, the index parameter in the HTTP request specifies the index in the event metadata.
The Default index configuration in a Splunk platform HEC destination	If the pipeline uses a Splunk platform HEC destination, and the Default index setting in the destination specifies an index name, then Ingest Processor sends data to that index.
The Default Index configuration in the HEC token	If the pipeline uses a Splunk platform HEC destination, and the Default Index setting in the HEC token specifies an index name, then Ingest Processor sends data to that index.
The Default Index configuration in the HEC global settings of a Splunk Enterprise deployment	If you're sending data to Splunk Enterprise using a Splunk platform HEC destination, and the Default Index setting in the HEC global settings of the Splunk Enterprise deployment specifies an index name, then Ingest Processor sends data to that index.
None of the previously described configurations specify an index	Ingest Processor sends data to the default index of the Splunk platform deployment, which is typically main. See Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual for more information.