Splunk Cloud Platform

Use Ingest Processors

System requirements for Ingest Processor

Before configuring Ingest Processor, make sure that the following requirements are met.

Software requirements

The Splunk Cloud Platform deployment where you want to run the Ingest Processor must meet or exceed the requirements listed in the Splunk Cloud Platform service description.

Additionally, the system clock of the host machine must be synchronized with a Network Time Protocol (NTP) server. If the system time is incorrect, this can cause the Ingest Processor solution to fail due to prematurely expired security tokens. For information about how to synchronize the system clock to an NTP server, refer to the documentation for your operating system.

Network requirements

Configure your firewall settings and the ports on your host machines to allow your Edge Processors to communicate with data sources, data destinations, the Edge Processor cloud service, and your Splunk platform deployment.

Firewall settings

The Ingest Processor solution must be able to communicate with the following external resources:

  • Any Splunk Cloud Platform deployments that are used as data destinations, including the deployment that is paired with your cloud tenant
  • Services that Splunk uses to monitor the health of the Ingest Processor solution and detect any unexpected disruptions in the service

The Splunk software collects information pertaining to the operational status of the Ingest Processor solution. This includes information such as the amount of data that is being sent through the Ingest Processor solution, as well as logs that track any events, warnings, or errors that have occurred.

This collected data only contains information pertaining to the operational status of the Ingest Processor solution. It does not contain any of the actual data that you are ingesting and processing through the Ingest Processor solution.

To allow the Ingest Processor solution to communicate with these external resources, make sure that your firewall allows access to the following URLs:

External resource URLs
The Ingest Processor solution Allow access to these URLs, where <tenant> is the name of your cloud tenant:
  • https://<tenant>.api.scs.splunk.com
  • https://<tenant>.auth.scs.splunk.com
  • https://auth.scs.splunk.com
  • https://beam.scs.splunk.com
The Splunk Cloud Platform deployment that is paired with your cloud tenant, as well as any deployments that are used as data destinations For each deployment, allow access to the following URL, where <deployment_name> is the name of the Splunk Cloud Platform deployment:


*.<deployment_name>.splunkcloud.com

Services that Splunk uses to monitor the health of the Ingest Processor solution Allow access to these URLs:
  • https://dataeng-data-cmp-prod.s3.us-east-1.amazonaws.com
  • https://http-inputs-products-telemetry.splunkcloud.com
  • https://telemetry-splkmobile.dataeng.splunk.com

Inbound ports

The Ingest Processor solution uses inbound ports to listen for data from data sources. Make sure that these ports are available and that your network policy allows them to be opened to incoming external traffic.

By default, the Ingest Processor solution is configured to use the following inbound ports to receive data:

Port Type of data received
8088 Data that's transmitted through HTTP Event Collector (HEC)
9997 Data from Splunk forwarders

The Ingest Processor solution supports the ingestion of syslog data, but do not have a default inbound port configured for it. You must choose the port number for receiving syslog data.

Outbound ports

The Ingest Processor solution use outbound ports to communicate with other components in your Splunk platform deployment and with external destinations. Make sure that these ports are available and that your network policy allows them to be opened to outgoing external traffic.

Port Details
443 the Ingest Processor solution uses port 443 to do the following:
  • Send data to Amazon S3.
9997 By default, Ingest Processor uses port 9997 to do the following:
  • Send internal logs to the Splunk Cloud Platform deployment that's connected to the tenant.
  • Send data to Splunk Cloud Platform indexes.

If your Splunk platform deployments use ports other than 9997 to listen for incoming data, then you must configure the Ingest Processor solution to use those ports instead and make sure that those ports are available.

  • During the first-time setup process, you connect your tenant to a Splunk Cloud Platform deployment. The listening ports used by the indexers in that deployment determine which ports the Ingest Processor solution to use to send internal logs. For more information, see First-time setup instructions for the Ingest Processor solution.
  • To configure the port that Ingest Processor uses to send data to a Splunk Cloud Platform index, start by adding a Splunk platform destination that specifies the correct port number. Then, use that destination in a pipeline and apply the pipeline in the Ingest Processor solution.

Subscriptions

There are two tiers of Ingest Processor licenses: Essentials and Premier.

The Ingest Processor Essentials tier is included with a Splunk Cloud Platform subscription, and accommodates a maximum Daily Processing Volume of 500 GB/day of incoming data.

The Premier tier is a priced SKU for Daily Processing Volumes over 500 GB/day of incoming data. For more information, contact your Splunk Sales representative.

For more information about licensing in Splunk Cloud Platform, see the Use the License Usage dashboards topic in the Splunk Cloud Platform Admin Manual.

For more information about Splunk Cloud Platform subscriptions, see the Subscription types section of the Splunk Cloud Platform Service Details topic in the Splunk Cloud Platform manual.

Last modified on 09 September, 2024
Manage users for the Ingest Processor solution   Ingest Processor pipeline syntax

This documentation applies to the following versions of Splunk Cloud Platform: 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release)


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters