Troubleshoot the Ingest Processor solution

Review this page if you are having difficulties with sending data through the Ingest Processor solution. If the problem that you're experiencing is not described on this page, you can find more information by doing the following:

• Review the list of known issues in the product. See Known issues.

If the problem persists, contact your Splunk representative for assistance. To help expedite the support process, you can generate a diagnostic report and send it to your Splunk representative.

My data is not being processed as expected

When you try to preview a pipeline, the preview results area displays a "No results" message or data that looks incorrect.

Alternatively, when you view the data that was sent from a pipeline to a destination, you notice that the data looks incorrect.

Cause

Reasons why a pipeline might not process data as expected include, but are not limited to, the following:

• The inbound stream of data is not being broken into events correctly. Data must be pre-processed into distinct events before being processed by a pipeline.
• The pipeline is not configured correctly.
• The pipeline preview is for the wrong destination.

Solution

For pipelines with multiple destinations, check to see if you are previewing the correct destination. If not, run the pipeline preview by selecting the Preview Pipeline icon () then select the destination name in the Preview drop-down list.

If this is not the case, make sure that event breaking and merging has been configured correctly for the source type of the data that you want to process.

1. Navigate to the Source types page.
2. Look for a source type with a name that matches the value of the sourcetype field in the data that you want to process.
   • If the source type exists, select it to view its configuration details. Confirm that the event breaking and merging behavior is configured correctly for the data that you want to process.
   • If the source type does not exist, then add it to the Ingest Processor service.

If the problem persists after you've verified the source type configuration, then complete the following steps to verify that the processing commands in your pipeline are configured correctly.

1. If you don't already have your pipeline open for editing, do the following:
   1. Navigate to the Pipelines page.
   2. On the Pipelines page, in the row that lists the pipeline you want to verify, select the Actions icon () and then select Edit.
2. From the side panel of the pipeline builder, select Sample data.
3. Enter or upload sample data that matches the inbound data that you want this pipeline to process, and then select Apply. You can use text strings that represent raw data or CSV values that represent parsed, field-extracted data. See Getting sample data for previewing data transformations for more information.
4. To generate a preview of what your data looks like after being processed by the pipeline, select the Preview Pipeline icon ().
5. Verify that the preview results match how you want the pipeline to process your data. If the results do not match, or the preview cannot be generated, then make sure that the SPL2 statement of your pipeline is written correctly and contains only supported SPL2 commands. See Ingest Processor pipeline syntax for more information.

Lookup dataset is not available

You created a lookup in the Splunk Cloud Platform deployment that is pair-connected with the Ingest Processor service, and then refreshed the scpbridge connection to bring that lookup into the tenant as a lookup dataset. However, when you try to work with this lookup dataset, you encounter one or more of the following problems:

• The Datasets page in the tenant does not include your lookup dataset.
• When you open the lookup dataset in the Search page and try to run a search, the search results pane displays an error or 0 results.
• When you use the Enrich events with lookups dialog box to configure a lookup for your pipeline, the Lookup dataset menu does not include your lookup dataset.

Cause

A permissions error is preventing you from fully accessing the lookup dataset. This problem can happen if your user account or the service account used by the scpbridge connection is missing read permissions for the following in Splunk Cloud Platform:

• The lookup table or definition
• The destination app that the lookup table or definition is associated with.

Solution

1. In Splunk Cloud Platform, select Settings, then select Lookups.
2. Select either Lookup table files or Lookup definitions, depending on how you created your lookup. If you're using a KV Store lookup, you must create a lookup definition for it.
3. In the row that lists your lookup, select Permissions.
4. Update the permissions as follows:
1. Set the Object should appear in option to All apps (system).
2. Make sure that Read permission is available to a role that is associated with your Splunk platform user account.
3. Make sure that Read permission is available to the role used by the service account. Typically, the name of this role is scp_user, if you used the role name suggested in Create a role for the service account during the initial setup of the Ingest Processor solution.
5. Make sure that a role that is associated with your user account and teh role used by the service account both have Read permission for the Destination app that is associated with the lookup.
1. Select Apps, then select Manage Apps.
2. Find the app that your lookup is associated with, and then select Permissions.
3. Select Read permission for the necessary roles, and then select Save.
6. Navigate to your Ingest Processor and then refresh the scpbridge connection.
1. Select the Settings icon () and tehn select System connections.
2. On the scpbridge connection, select the Refresh icon ().

Lookup dataset fails to download or update

When you try to apply a pipeline that contains the lookup command, the Ingest Processor enters the Error status and you see the following error message, where <lookup_dataset> is the name of your lookup dataset:

max lookup size exceeded on initial lookup download for dataset <lookup_dataset>

Alternatively, you've already successfully applied the pipeline to your Ingest Processor, but subsequent updates to the lookup dataset are not being reflected in the processed data.

Cause

The lookup dataset exceeds the maximum supported size of 200 MB, so the Ingest Processor cannot download and use it. If the lookup dataset was under 200 MB when the pipeline was initially applied but exceeded the maximum size after changes were made to the dataset, then the Ingest Processor fails to download and use the updated dataset but continues to process incoming data using the prior version of the dataset.

Solution

1. Reduce the size of the lookup table associated with the dataset until it is under 200 MB. For information about updating lookup datasets and synchronizing the information between Splunk Cloud Platform and the Ingest Processor, see Update lookup datasets.
2. Do one of the following:
• If the pipeline failed to apply initially, then try to apply it again.
• If the pipeline is already applied, then wait for the Ingest Processor to download the updates to the lookup dataset.