Use templates to create pipelines for Edge Processors

To help you get started on creating and using pipelines, the Edge Processor solution includes sample pipelines called templates. Templates are Splunk-built pipelines that are designed to work with specific data sources and use cases. Templates include sample data and preconfigured SPL2, so you can use them as a starting point in order to build custom pipelines to solve specific use cases or as a reference to learn how to write SPL2 to build pipelines.

To view a list of the available pipeline templates, log in to your tenant, navigate to the Pipelines page, and then select Templates.

To create a pipeline using a template, complete the following steps.

Prerequisites

Before starting to create a pipeline, make sure that the destination that you want the pipeline to send data to is listed on the Destinations page of your tenant. If your destination is not listed on that page, then you must add that destination to your tenant. See Add or manage destinations for more information.

Steps

1. Navigate to the Pipelines page and then select New pipeline.
2. Select the pipeline template that you want to use, and then select Next.
3. Specify a subset of the data received by the Edge Processor for this pipeline to process. To do this, you must define a partition by completing these steps:
1. Select the plus icon () next to Partition or select the option that matches how you would like to create your partition in the Suggestions section.
2. In the Field field, specify the event field that you want the partitioning condition to be based on.
3. To specify whether the pipeline includes or excludes the data that meets the criteria, select Keep or Remove.
4. In the Operator field, select an operator for the partitioning condition.
5. In the Value field, enter the value that your partition should filter by to create the subset. Then select Apply. You can create more conditions for a partition in a pipeline by selecting the plus icon ().
6. Once you have defined your partition, select Next.
4. Templates include sample data, so you can review it and then select Next.
5. Select the name of the destination that you want to send data to. Then, do the following:
1. If you selected a Splunk platform S2S or Splunk platform HEC destination, select Next.
2. If you selected another type of destination, select Done and skip the next step.
6. (Optional) If you're sending data to a Splunk platform deployment, you can specify a target index:
1. In the Index name field, select the name of the index that you want to send your data to.
2. (Optional) In some cases, incoming data already specifies a target index. If you want your Index name selection to override previous target index settings, then select the Overwrite previously specified target index check box.
3. Select Done.

Be aware that the destination index is determined by a precedence order of configurations. See How does an Edge Processor know which index to send data to? for more information.

The pipeline builder displays the pipeline that you've just created based on the selected template and configuration options. You can review the comments in the pipeline editor to learn more about what the pipeline will do to the sample data.

7. To save this template as a pipeline that you can apply to Edge Processors, do the following:
   1. Select Save pipeline.
   2. In the Name field, enter a descriptive name for your pipeline.
   3. Select Save.
8. To apply this pipeline to an Edge Processor, do the following:
   1. Navigate to the Pipelines page.
   2. In the row that lists your pipeline, select the Actions icon () and then select Apply/remove.
   3. Select the Edge Processors that you want to apply the pipeline to, and then select Save.

   You can only apply pipelines to Edge Processors that are in the Healthy status.

It can take a few minutes for the Edge Processor service to finish applying your pipeline to an Edge Processor. During this time, all Edge Processors that the pipeline is applied to enter the Pending status. To confirm that the process completed successfully, do the following:

• Navigate to the Edge Processors page. Then, verify that the Instance health column for the affected Edge Processors shows that all instances are back in the Healthy status.
• Navigate to the Pipelines page. Then, verify that the Applied column for the pipeline contains a The pipeline is applied icon ().

You might need to refresh your browser to see the latest updates.

The Edge Processor that you applied this pipeline to can now process the data it receives based on the processing instructions defined in the template.