Using source types to break and merge data in Ingest Processor

The source type is one of the default fields that Splunk software assigns to events. It identifies the kind of data that you are working with and indicates the original source of the data.

In Ingest Processor, you can create source type configurations and use them to specify the following behavior:

• How Ingest Processor breaks and merges the inbound stream of data into distinct events. The event breaking and merging operations defined in your source type configurations are applied to inbound data if it meets the following criteria:
  • The sourcetype value of an event matches the name of a source type configuration in the Ingest Processor service.
  • The inbound data isn't already event-broken through other means, such as by the EVENT_BREAKER configuration in a universal forwarder.
• What data a pipeline processes. When you create a pipeline, it selects a subset of data from the all_data_ready in the Ingest Processor to be processed based on your source type selection. The pipeline processes only the events that have a matching sourcetype value.

When creating a pipeline, you can combine your selected source type with other conditions to choose a more specific subset of data to process. See Partitions for more information.

By default, the Ingest Processor service includes event breaking and merging configurations for a variety of common source types. See Automatically recognized source types in the Splunk Cloud Platform Getting Data In manual for a list of default source types. If the source type that you want to work with is not listed, then you must add and configure that source type in the Ingest Processor service. You can also edit the default source types to meet your needs.