Add source types for Edge Processors

In the Edge Processor service, source type configurations are used to do the following:

• Break and merge the inbound stream of data into distinct events.
• Specify what data a pipeline processes.

If the source type that you want to work with is not included by default, then you can add and configure it in the Edge Processor service.

When you add a source type, you configure the following options. Each option is equivalent to a property that is supported in props.conf files:

Edge Processor option	props.conf property
Line breaking	LINE_BREAKER
Merge lines into events	SHOULD_LINEMERGE
Multiline event delimiter	BREAK_ONLY_BEFORE
Maximum lines per event	MAX_EVENTS

For more information about props.conf, see props.conf in the Splunk Enterprise Admin Manual.

If you already have a props.conf file with the configurations that you want to use, you might be able to reuse those configurations by copy-pasting them into the source types in the Edge Processor service. However, you must ensure that all of the regular expressions in these configurations use valid Regular Expression 2 (RE2) syntax. For more information about the regular expression syntax that the Edge Processor solutions supports, see Regular expression syntax for Edge Processor pipelines.

This is step 3 of 6 for using an Edge Processor to process data and route it to a destination. To see an overview of all of the steps, see Quick start: Process and route data using Edge Processors.

Prerequisites

Before you can add a source type in the Edge Processor service, you must know the exact name of the source type that you want to work with. This source type name must be identical to the value of the sourcetype field in the data that you want to process and configure event breaking for.

Steps

1. On the Source types page, select New source type.
2. In the Name field, enter the exact name of the source type that you want to work with. The source type name must meet these requirements:
   • The name must be unique. If you want to override a source type configuration that already exists in your tenant, you must either edit the existing source type configuration or rename it so that you can define a new configuration using the original source type name.
   • The name cannot be splunk-edge-processor-metrics or splunk-edge-processor-log. These are reserved for internal use only.
3. In the Line breaking field, specify the delimiter that indicates the end of one event and the start of another. If using a line break as the delimiter meets your requirements, then leave this field at the default value of ([\r\n]+). Otherwise, enter a different RE2 capture group that matches the delimiter.

   This delimiter gets dropped from your data. It is treated as something that exists between events rather than something that is part of an event. For more information, see the description of the LINE_BREAKER property in props.conf in the Splunk Enterprise Admin Manual.

4. If your inbound data consists of multiline events, then do the following:
   1. Select Merge lines into events.
   2. In the Multiline event delimiter field, enter an RE2 expression that matches the start of each multiline event.
   3. (Optional) To specify the maximum number of lines to include in a single multiline event, expand Advanced settings and enter your desired maximum number of lines in the Maximum lines per event field.
5. (Optional) To generate a preview that shows how your source type configuration breaks and merges inbound data streams into events, do the following:
   1. Select the Edit sample data icon ().
   2. In the Edit sample data dialog box, enter or upload sample data for generating the preview.

      The sample data must be in the same format as the actual data that is associated with the source type. See Getting sample data for previewing data transformations for more information.

   3. Select Save.
   4. Select the Run To Preview Source Type icon () to generate the preview. Use the preview results to validate your source type configuration.
6. To save your source type, do the following:
   1. Select Save source type.
   2. (Optional) In the Description field, enter a description for the source type.
   3. Select Save.

You now have a source type configuration that breaks and merges any inbound data that has a matching sourcetype value.

You can also use this source type in your pipelines. For information about creating pipelines and applying them to Edge Processors, see Create pipelines for Edge Processors and Apply pipelines to Edge Processors.