Data format requirements for Splunk IAI
Splunk Industrial Asset Intelligence (IAI) requires that two types of data produced by your industrial assets be compatible with the IoT common information model described in this documentation:
- Metrics data that can be aggregated or displayed as a time series. Metrics data must be stored in a metrics index and map to the IoT common information model for metrics.
- Alarms produced by an asset, device, or sensor when a threshold is reached. Alarm data must be stored in an event index and map to the IoT common information model for alarms.
Alarms are a subset of event data. Your assets may also produce other event data, such as log data. Splunk IAI does not have any requirements for event data that is not alarm data.
This common information model exists to make sure that data from various types of sensors and IoT devices can be analyzed and monitored together. Some use cases in the Splunk platform have no requirement that your data match a particular schema, but normalizing similar types of data to a common information model can make visualizing and searching that data easier.
Metrics and events indexes
When Splunk Enterprise ingests data, it stores the data either in a metrics index or an event index. Each index type is optimized for storage and retrieval of that data. Your Splunk Enterprise administrator is responsible for setting up metrics and event indexes to store the data you will monitor and analyze in Splunk IAI.
When you work with your Splunk Enterprise administrator on setting up data ingestion from your industrial assets, ensure that the metrics data is routed to and stored in metrics indexes, and that alarm data is routed to and stored in event indexes.
IoT common information model
This information model adds additional requirements for data that you plan to monitor and analyze with Splunk IAI.
Requirements for metrics data from your industrial assets
The Splunk platform supports metrics data that matches a schema in which each metric contains a timestamp, a metric name, a value, and at least one dimension field. In addition to the fields that the Splunk platform metrics schema requires for all metrics data, the IoT common information model has three additional requirements:
metric_namevalue must not contain dot notation.
- Metrics data points must contain one required dimension field:
- Both the
assetfields must be set at index time.
The IoT common information model also supports several other dimension fields, but they are not required.
This table lists the required and optional fields for metrics data in the IoT common information model:
||time||Required field for all metrics data.||The timestamp of the metric in UNIX time notation.||2017-08-14 17:12:39.000|
||string||Required field for all metrics data.||The numeric value of the metric. This field is a 64-bit floating point number, which supports precision between 15 and 17 decimal digits.||42.12345|
||string||Required dimension field for IAI.||Represents the name of the asset, device, or sensor that is generating or monitoring the metric. To facilitate data association in Splunk IAI, you can use dot notation to describe the full path to the asset as defined by your asset hierarchy, but this is not required. See Model your asset hierarchy in Splunk IAI.||Factory A.Line 1.WoodGrinder A|
||string||Optional dimension field for IAI.||Quality associated with the generated metric.||"Good", "Bad", "Any other string representing quality."|
||string||Required field for all metrics data.||The name of the metric. In Splunk IAI, the
||temperature, speed_mph, weight|
||string||Optional dimension field for IAI.||Type of metric. Defaults to "gauge", the only supported type of metric.||gauge|
||string||Optional dimension field for IAI.||Captures the status of the asset when the metric was generated.||alarm_state, resolution_state|
||string||Optional dimension field for IAI.||Unit of the metric.||ft, yd, cm, pt, qt|
Requirements for alarm data from your industrial assets
To function as expected in Splunk IAI, alarm data must contain some fields required by the IoT common information model, however the names of the fields are suggestions, not requirements. For example, each alarm event must have an alarm name, but that field does not need to be called
Each alarm must be a unique event.
||time||Optional||Alarm acknowledgment time.||1550874434|
||string||Required||Code associated with the alarm message or name of the alarm.||Low|
||string||Required||The name of the asset, device, or sensor generating or monitoring the alarm.||Tire 7|
||string||Optional||Category or group of the alarm.||Truck Fleet|
||string||Optional||Alarm message.||Low pressure alert|
||number||Optional||Severity of the alarm.||1,2,3|
||time||Optional||Time when the alarm generation started. Can be aliased from
||string||Optional||State of the alarm.||Active|
||time||Optional||Time when the alarm stopped.||1550874814|
||string||Optional||Type of alarm.||Condition|
See Get your metrics and alarm data in to Splunk IAI for information on which ingestion methods handle mapping your alarm data to the IoT common information model for you.
Searching your industrial asset data in the Splunk platform
In addition to using Splunk IAI to monitor and analyze your data, you can run searches in the Search & Reporting App. If you are not familiar with the Search Processing Language (SPL), start by working through the Splunk Enterprise Search Tutorial, which walks you through adding sample data, running searches, and creating simple dashboards and reports.
Searching metrics data is different than searching event data. Metric searches retrieve raw metrics from a Splunk metrics index without any additional processing. To learn how to search metrics data in the Splunk platform, see Search and monitor metrics in the Splunk Enterprise Metrics manual.
- To learn about getting data in to Splunk IAI, see Get your metrics and alarm data in to Splunk IAI.
- For an introduction on how the Splunk platform supports metrics data, see Overview of metrics in the Splunk Enterprise Metrics manual.
- For a description of the Splunk platform metrics schema, see Metrics data format in the Splunk Enterprise Metrics manual.
- For more information about using the Search & Reporting App and the Search Processing Language, see Get started with Search in the Splunk Enterprise Search Manual.
Splunk IAI terminology
Support and resources for Splunk IAI
This documentation applies to the following versions of Splunk® Industrial Asset Intelligence (Legacy): 1.1.0, 1.1.1, 1.2.1, 1.2.2, 1.3.0