Splunk® Enterprise

Data Model and Pivot Tutorial

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

About data models and data model objects

The topics in this chapter show you how to use the Data Model Builder to design and build data models for the tutorial data.

What is a data model?

A data model is a hierarchically-structured search-time mapping of semantic knowledge about one or more datasets. It encodes the domain knowledge necessary to build a variety of specialized searches of those datasets. Briefly put, data models generate searches. These specialized searches are in turn used to generate reports for Pivot users.

To create an effective data model, you must understand your data sources (whether it's derived from a log file, TCP/UDP network input, received from a scripted input for an API, and so on) and your data semantics (how the various fields in your data are extracted, related, and organized). This information can affect your data model architecture.

Data models can get their fields from extractions that are defined on the Splunk Web Settings > Fields > Field extractions page or, for Splunk Enterprise, by editing the props.conf and transforms.conf files. But when you define your data model, you can also arrange to have it get additional fields at search time through regex-based field extractions, lookups, and eval expressions.

In this tutorial, your data sources are web access and secure log files. Most of the fields are automatically extracted. Other fields will be added using lookup files and calculated with eval expressions.

About data model objects

Data models are composed of one or more objects. Each object is a dataset that corresponds in some manner to a set of data in your index. Objects break down into four types: Events objects, search objects, transaction objects, and child objects.

Objects in data models can be arranged in parent/child relationships. Each top-level or root object can have child objects which inherit the constraints and attributes of the parent and have additional constraints and attributes of their own.

Note: Data model objects are a category of knowledge object. However, data model objects often use other knowledge objects such as extracted fields, calculated fields, and lookups to define the specfic sets of data that they represent.

Here is an example of a data model as viewed through the Data Model Builder.

6.4.0 DM Builder forPivotTutorial.png

In this example, the object hierarchy is in the left-hand sidebar. The Splunk Server root event object is selected. The Splunk Server object contains all of the data in the data model. The child objects that branch off of the Splunk Server object (such as Scheduler, Acceleration, and Licenser) each contain different subsets of that data.

On the right side of the Data Model Builder are the object constraints that define the dataset represented by the object, and the list of attributes associated with the object. The other topics in this chapter show you how to create a data model and then use the Data Model Builder to define its object hierarchies and object attributes.

Object constraints

All data model objects are defined by sets of constraints that filter out events that aren't relevant to the object; they help to define the dataset that the object represents. A typical constraint looks like the first part of a search, before pipes and additional search commands are added.

Constraints are inherited by child objects to ensure that each child object represents a subset of the data represented by its parent objects. Pivot users can then use these child objects to design reports with datasets that already have extraneous data prefiltered out.

Object attributes

An object's attributes are a set of fields associated with the dataset that the object represents. Object attributes come in five flavors: Auto-extracted, Eval expression, Lookup, Regular Expression, and Geo IP.

Object attributes are inherited. A child object will automatically have all of the attributes that belong to its parent. You can design a relatively simple data model where all of the necessary attributes for a specific object tree are defined in its root object, and the child objects would be differentiated from the root object and from each other only by their constraints.

Attributes serve several purposes. Their most obvious function is to provide the set of fields that Pivot users work with to define and generate a pivot report; the set of fields they have access to is determined by the object they choose when they enter Pivot. You might add attributes to a child object to provide fields to Pivot users that are specific to the dataset covered by that object.

Learn more about data models

The information discussed in this topic is limited to what you need to know to build the data models for the tutorial data. For more information, see About data models and Design data model objects in the Knowledge Manager Manual.

Next steps

Proceed to the next topic, where you will create a new data model.

Add lookup files
Create a new data model

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11


@Pyro wood: I have updated the topic with an example of a data model, as viewed through the Data Model Builder. It is one of the example data models delivered with the product.

Mness, Splunker
January 13, 2016

Hello Splunk-Team,
as a splunk beginner I had difficulties to understand how a data model should look like when its done or how its designed. It would be really helpful if there could be a picture of an example data model so new users are able to comprehend the given information. :)

Pyro wood
January 12, 2016

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters