Splunk® Data Stream Processor

Function Reference

Acrobat logo Download manual as PDF

Acrobat logo Download topic as PDF

Union

This topic describes how to use the function in the Splunk Data Stream Processor.

Description

Combines streams with the same input schema into one stream with all of the events of the input streams. If the combined streams do not have the same schema, an error is shown.

Function Input/Output Schema

Function Input
collection<collection<record<R>>>
This function takes in multiple data streams where each stream has schema R.
Function Output
collection<record<R>>
This function outputs a single data stream with schema R.

Syntax

The required syntax is in bold.

union
<DataStream> ["," <DataStream>...]

Required arguments

DataStream
Syntax: <expression>
Description: The data stream you want to perform the union on.

Usage

If you are using the Union function in the Canvas Builder, click on the View Configuration button and select the function on the data stream branch that you'd like to union.
If you are using the Union function in the SPL2 Builder, you'll need to use SPL2 variables.

Using SPL2 variables to union data streams in the SPL2 Pipeline Builder

When constructing a pipeline using the SPL2 Pipeline Builder and performing a union, you must use SPL2 variables to properly union your data streams. SPL2 variables allow you to store branches of your data streams as SPL2 statements. Variable names must begin with a dollar sign ($) and can only contain letters, numbers, or underscores. You must have the same number of variables as branches in your data stream. For example, if you want to union three branches together, you will need three SPL2 variables.

You can store data streams as SPL2 variables and union these data streams like this:

$my_variable1 = | from read_from_aws_cloudwatch_metrics("4e1a64d8-0849-4324-9298-1e655ea4ba87") | eval sourcetype="cloudwatch"; 
$my_variable2 = | from forwarders("forwarders:all") | eval sourcetype="forwarders";

|from $my_variable1 | union $my_variable2 | into index("", "metrics");

In this example, you have two variables representing the two data stream branches in your pipeline. The first variable stores the first branch of your pipeline containing a Amazon CloudWatch source function and an eval function. The second variable stores the second branch of your pipeline containing a Forwarders Service source function and a different eval function. After defining your variables, you can then reference them in your SPL2 statements and union the two branched data streams together. This screen image shows two data streams from two different data sources being unioned together in a pipeline.

SPL2 example

This example unions together two branches, one branch containing the data from the Ingest Service and another branch containing the data from the Forwarders Service.

$forwarder_events = | from forwarders("forwarders:all");
$ingest_api_events = | from ingest_rest_api("rest-api:all");
| from $forwarder_events | union $ingest_api_events | into index("", "main");
Last modified on 06 November, 2020
PREVIOUS
To Splunk JSON
  NEXT
Where

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters