Improving data ingestion using the Edge Processor solution

The Edge Processor solution is a data transformation service within Splunk Cloud Platform. Use the Edge Processor solution to filter, mask, and transform data before routing that data to its supported destinations.

The Edge Processor solution is designed to provide you with more data manipulation abilities than Splunk Cloud Platform alone, and you can set it up through a simple installation process. The Edge Processor solution also allows you to view your inbound and outbound data volumes through a UI-based control plane called the Edge Processor service.

Use the Edge Processor service to also configure Edge Processors. After you configure your Edge Processors, install them in your local environment. You can then use the Edge Processor service to define data sources and destinations for your Edge Processors.

Create Edge Processor pipelines to define the logic for filtering, masking, and transforming data. Apply pipelines to any number of Edge Processors. To get started with the Edge Processor solution, see Get started with the Edge Processor solution in the Use Edge Processors manual.

Edge Processors can receive data from sources including:

• Splunk universal forwarders
• Heavyweight forwarders
• HTTP clients and logging applications through the HTTP Event Collector (HEC)
• Syslog
• Splunk Connect for Syslog (SC4S)

Edge Processors can route data to destinations including:

• Splunk Enterprise
• Splunk Cloud Platform
• Amazon S3

Compare Ingest Actions to the Edge Processor solution

Ingest Actions is another Splunk data transformation service. Ingest Actions and the Edge Processor solution can largely handle the same use cases. For example, both allow you to filter verbose data sources, such as Windows event logs, to retain selected events or content within an event. Both the Edge Processor solution and Ingest Actions let you match a certain event code, mask the extensive message field at the end of Windows events, and route an unfiltered copy of data to an AWS S3 bucket.

The Edge Processor solution offers a centralized control plane to manipulate your data pipelines through Search Processing Language, version 2 (SPL2) while Ingest Actions offers a graphical user interface over existing props and transforms so that you can create rulesets to affect the data transformation. The following table provides a side-by-side comparison of the two services:

	Edge Processor solution	Ingest Actions
Platform availability	Is available only in Splunk Cloud Platform.	Is natively available in both Splunk Enterprise and Splunk Cloud Platform. This is with the exception of the add-on for Google Cloud Platform (GCP) in the Splunk Cloud Platform.
Cost	All current Edge Processor features are free to all Splunk Cloud users.	All current Ingest Actions features are free to all Splunk Enterprise and Splunk Cloud users.
Method of access	Requires activation. Ask a Splunk sales representative for access to the Edge Processor solution if you are already a Splunk Cloud Platform user.	Is natively available in both Splunk Enterprise and Splunk Cloud Platform.
Transformation capabilities	Relies on Splunk Search Processing Language, version 2 (SPL2), which allows you to create tightly defined logic to transform data through pipelines.	Transforms data through rulesets, which are defined through drop-down menu options, offering more ease of use but less detailed options.
Closeness to the data source	Is usually closer to the data source when you transform your data. It represents another forwarding tier.	Is farther away from the data source if you configure it directly on the indexing tier. If you configure Ingest Actions on the heavyweight forwarding tier, it is equally close to the data source as the Edge Processor solution.
User interface	Has a graphical user interface (UI) and allows you to compare your inbound and outbound data. For example, you can preview what percentage of your inbound data becomes your outbound data based on how you code your pipeline logic. You can also see all your Edge Processors in one place and deploy your pipeline logic to your different Edge Processors in one place.	Has a graphic user interface (UI) and includes data previews before implementing your code. You can visualize directly how events are edited before and after you deploy your ruleset. However, your Ingest Action rulesets might not be visible all in one place. Your Ingest Actions ruleset are available on the indexing or heavyweight forwarding tier that you implemented them on.
Sources	Can receive data from these sources: Splunk universal forwarders Heavyweight forwarders HTTP clients and logging applications through HEC Syslog Splunk Connect for Syslog (SC4S)	Can receive data from any source supported by the Splunk platform. You cannot deploy Ingest Actions on a universal forwarder, but you can receive data from a universal forwarder. You can deploy Ingest Actions on a heavyweight forwarder.
Destinations	The Edge Processor solution can route to the same destinations as Ingest Actions: Amazon S3 Splunk Enterprise Splunk Cloud Platform	Ingest Actions route to the same destinations as the Edge Processor solution: Amazon S3 Splunk Enterprise Splunk Cloud Platform