Splunk Cloud Platform

Use Ingest Processors

Verify your Ingest Processor and pipeline configurations

The Ingest Processor service starts processing and routing your data after you've completed the following steps:

After you complete these steps, the Ingest Processor service processes data and sends it to a destination based on the data processing instructions defined in the applied pipelines.

To confirm that data is actually flowing through your Ingest Processor pipeline, you can view the inbound and outbound data metrics of the Ingest Processor. As an additional confirmation step, you can verify your data at its destination. For example, you can search an index to confirm that your data is reaching that index as expected. See the sections that follow for more detailed guidance on verifying that your Ingest Processor pipeline is working as expected.

View the inbound and outbound data metrics of an Ingest Processor

In the Ingest Processor service, you can open a detailed view of your Ingest Processor that displays information such as the amount of data that your Ingest Processor is receiving and sending out to destinations.

  1. Navigate to the Ingest Processor page.
  2. In the row that lists your Ingest Processor pipeline, select the Actions icon (Image of the Actions icon) and then select Open.
  3. View the Inbound data and Outbound data values to confirm that data is flowing through your Ingest Processor pipeline.

If the data flow metrics do not match what you expect, then verify your configurations.

Search for your data in the destination index

Use Splunk Cloud Platform to search for the data that you sent through your Ingest Processor.

  1. Log in to the Splunk platform deployment that you configured your Ingest Processor to send data to.
  2. From the Apps panel in Splunk Web, select Search & Reporting.
  3. Search the destination index to confirm that it contains the expected events. For example, if you configured your Ingest Processor pipeline to send data to an index named my_index, then use the following search criteria to find your data:
    index="my_index"

If your processed data is not showing up at its destination as expected, then verify your configurations.

Confirming and troubleshooting your configurations

If you encounter unexpected results or behavior while using the Ingest Processor solution, make sure that your data source, source type, pipeline, and destination are configured correctly. Specifically, verify the following:

  • If you're working with data that is transmitted through HTTP Event Collector (HEC), make sure that the HTTP requests for sending the data are formatted correctly.
  • The source type of the data that you want to process is listed on the Source types page in the Ingest Processor service, and this source type is configured with the appropriate event-breaking definitions.

When a source type configuration is opened for editing, you can generate a preview that confirms how that configuration breaks and merges the inbound data stream into events. See the Getting sample data for previewing data transformations and Add source type for Ingest Processor topics in this manual for more information.

  • Your pipeline is configured correctly. Make sure that your pipeline isn't filtering out data that you want to keep.

When your pipeline is opened for editing, you can generate a preview for each destination to confirm how your pipeline processes data. See Getting sample data for previewing data transformations and Create pipelines for Ingest Processors for more information.

  • The destination used by your pipeline is configured with the correct connection settings and credentials.
  • If you're sending data from to the Splunk platform through HEC, make sure that your HEC token and index configurations are not being overridden by a configuration that's higher in the precedence order.

If the problems persist, do the following:

  • Review the logs for your Ingest Processor pipeline and the associated supervisor to identify the cause of the problem.
  • Review the troubleshooting documentation for potential solutions or workarounds.
Last modified on 17 July, 2024
Send data from Ingest Processor to Amazon S3   View data flow information about an Ingest Processor pipeline

This documentation applies to the following versions of Splunk Cloud Platform: 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release)


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters