Get started with metrics

The Splunk platform gathers metrics from different sources and stores this data into a new type of index that is optimized for ingestion and retrieval of metrics.

The Splunk platform supports the following metrics-gathering tools natively:

• The collectd agent, a Unix-based daemon, with the write_HTTP plugin. Collectd supports over 100 front-end plugins.
• The StatsD line protocol, which is used by a wide range of client libraries and other open source tools.

Both of these tools are lightweight and easy to use, and they have a large community of support. If you want to start gathering performance metrics from your applications and systems, review these tools to determine whether either of them suits your environment.

If you prefer to use a different metrics-gathering tool, you can still use the Splunk platform to collect and analyze your data with manual configuration.

Metrics data format

Metrics data uses a specific format with the following fields.

Field	Required	Writable or Internal	Description	Example
_time	X	Writable	The timestamp of the metric in UNIX time notation.	2017-08-14 17:12:39.000
metric_name:<metric_name>	X	Writable	The metric name. It always has a numeric value. This is a 64-bit floating point number, which supports precision between 15 and 17 decimal digits.	metric_name:os.cpu.user=42.12345
<dimension 0> ... <dimension n>	X	Writable	An arbitrary number of dimension fields that indicate how metrics can be split.	ip
_dims	X	Internal	An auto-generated internal field that contains the names of all of the dimensions in the metric event. The purpose of this field is to return a list of unique dimension names in a metrics index.	_dims::ip
source		Internal	The source of the metrics data.	udp:8125
host	X	Internal	The origin host. A standard field in Splunk software.	server007
index	X	Internal	The metrics index name. A standard field in Splunk software.	metricsindex
sourcetype	X	Internal	The data structure of the metric. A standard field in Splunk software.	statsd

Supported line protocols

Metrics in the Splunk platform natively supports the following metric line protocols:

• Plain StatsD over UDP/TCP
• The StatsD extension with dimensions over UDP/TCP
• Collectd over HTTPS using HTTP Event Collector (HEC)

For details about getting data in, see Get metrics in from StatsD and Get metrics in from collectd.

To support other line metric protocols, you can use custom transformations to get metrics data into Splunk platform from other tools. For details, see Get metrics from other clients.

Metrics source types

The Splunk platform includes the following pre-trained source types to support the most widely-supported line metric protocols:

Source type name	Description
statsd	Supports data using the metric line protocols for plain StatsD and the StatsD extension with dimensions.
collectd_http	Supports data using the metric line protocol for collectd.
metrics_csv	Supports data in CSV format. For usage details, see Get metrics in from other sources.

Metrics indexes

To store and analyze metrics data as efficiently as possible, metrics data is stored in a type of index just for metrics. Metrics indexes store metric data points in a format that provides faster search performance and more efficient data storage than you will find with events in event indexes.

A metrics index can be used only for metrics data. You cannot convert an events index to a metrics index, or vice versa.

If you use Splunk Enterprise, see Create metrics indexes in the Managing Indexers and Clusters of Indexers manual.

If you use Splunk Cloud, see Manage Splunk Cloud Platform indexes in the Splunk Cloud Platform Admin Manual.

For information about how metrics data is metered, see How Splunk Enterprise licensing works in the Admin Manual.

Internal metrics indexes

Use the _metrics index to monitor internal metrics for your Splunk Enterprise implementation. It is a metrics analog of the _internal event index.

The _metrics_rollup index is an internal index that is designed for use by the Monitoring Console. Data flows to it only when the Monitoring Console is enabled and the metrics rollup policy configured in the context of the Monitoring Console app is also enabled.

To learn how to enable data to flow to the _metrics_rollup index, see Resource Usage: CPU Usage in Monitoring Splunk Enterprise and find the subsection on the Median Historical CPU Usage dashboard.

Assign default metrics indexes to user roles

You can assign default metrics indexes to user roles. See Add and edit roles with Splunk Web in Securing Splunk.

When you run a search with metrics commands such as mcatalog or mstats and you do not filter the search by a specific index, the search automatically searches the default indexes assigned to your role. If you run a metrics search that does not filter by a specific metrics index and you have no default metrics indexes assigned to your role, the metrics search runs over an empty dataset.

Search and CLI commands with metrics

• To analyze metric data and enumerate items in a metrics index, use the mstats and mcatalog search commands.
• The msearch command enables you to view individual metric data points, without aggregation.
• The mcollect and meventcollect commands convert event log data into metric data points at search time.

Other search commands that work with events do not work with metrics. For example, the delete command does not work with metrics. For more about searching a metrics index, see Search and monitor metrics.

Administrative CLI commands may not all work with metrics. You can use commands such as add index and list index with metrics when using the -datatype metric parameter. See Create metrics indexes in the Managing Indexers and Clusters of Indexers manual.