Use Datastreamer to load data into your Splunk platform deployment

Use Datastreamer to send your data to your Splunk platform deployment from your Edge Hub. To do this, load data into your Splunk platform instance using the Data Ingestion page in Splunk Web.

Once you have created a Datastreamer, you can edit or delete it, or send data to a metrics index or configure Datastreamer settings.

Create a Datastreamer

Follow these steps to set up and configure your Datastreamer.

Prerequisites

• Confirm that your Edge Hubs are updated to firmware version 2.2. In order for your Edge Hubs to utilize datastreamers, they must be updated to the latest firmware version.

Steps

1. In Splunk Web, navigate to the OT Intelligence Splunk Application.
2. Select the Data Ingestion tab.
3. In the Data Ingestion tab, select the Create Datastreamer button.
4. Enter the name of the Datastreamer. This should be an easily identifiable name that describes the Datastreamer's purpose. For example, network traffic.
5. Enter the HEC hostname. The HEC hostname is the url that the Datastreamer will send data to and is specific to the configuration for your Splunk platform instance.
6. Configure your Datastreamer by selecting the following options:

Option	Description
HEC token	A dropdown list of the HECs created under Data Inputs. The HEC token is specific to the configuration for your Splunk platform instance.
Index	The index that is assigned to your HEC token that the Datastreamer is sending data to.
Topics	Select a list of pre-filled MQTT topics or define your own MQTT topic that the Datastreamer will subscribe to.
Data Volume	Select Standard or High. If the user anticipates a large throughput of data, the user should select High and * more worker threads are applied to that Datastreamer.
Backlog Size	The allocated size of the database backlog for this Datastreamer. If the Datastreamer can't send data to the Splunk platform instance, it will save the entry into the database.
Batch Size	The number of data events that must arrive at the Datastreamer before sending to Splunk platform.
Source Type	The source type that will be attached to the data sent by this Datastreamer. If you configure a Datastreamer to send data to a metrics index, you must set the Source Type to log2metrics_json in the Data Ingestion tab.

Recommended Datastreamer configuration settings

The following are the recommended settings to stream all types of data from Edge Hub.

Sensors and Anomalies

Option	Value
Name	sensors & anomalies
Index	A metrics index, suffixed with _data.
Topics	edgehub/+/+/values, edgehub/+/+/anomalies
Data Volume	High
Backlog Size	3,000,000
Batch Size	10 or 25
Source Type	log2metrics_json

Status

Option	Value
Name	status
Index	A metrics index, suffixed with _status.
Topics	edgehub/health
Data Volume	Standard
Backlog Size	100,000
Batch Size	10
Source Type	log2metrics_json

Logs

Option	Value
Name	logs
Index	An events index
Topics	edgehub/logs
Data Volume	Standard
Backlog Size	100,000
Batch Size	10

OPCUA

Option	Value
Name	opcua
Index	An events index
Topics	edgehub/opcua
Data Volume	Standard
Backlog Size	100,000
Batch Size	1 or 10

Modbus

Option	Value
Name	modbus
Index	An events index
Topics	edgehub/modbus
Data Volume	Standard
Backlog Size	100,000
Batch Size	1 or 10

SNMP

Option	Value
Name	snmp
Index	An events index
Topics	edgehub/snmp
Data Volume	Standard
Backlog Size	100,000
Batch Size	1 or 10

SDK

Option	Value
Name	sdk
Index	An events index
Topics	edgehub/sdk/<topic that SDK client is sending>
Data Volume	Standard
Backlog Size	100,000
Batch Size	1 or 10

External MQTT

Option	Value
Name	mqtt
Index	An events index
Topics	edgehub/mqtt_events/<topic that is being sent to MQTT>
Data Volume	Standard
Backlog Size	100,000
Batch Size	1 or 10

Edit or delete a Datastreamer

To edit the fields of a Datastreamer, in the Data Ingestion tab, select a row of any existing Datastreamers. You can't edit the Backlog Size field.

You can also delete a Datastreamer by selecting the Datastreamer and clicking on the garbage can icon.

Migrate from Splunk App for Edge Hub to Datastreamer

If you are using an older version of Splunk OT Intelligence (previously called Splunk App for Edge Hub), you can easily migrate settings to the Datastreamer. If you encounter issues migrating settings from Splunk App for Edge Hub to Datastreamer, manually configure your settings in the Data Ingestion tab.

In the Data Ingestion tab, click the Migrate button to automatically convert settings to the Datastreamer format.