Splunk® Enterprise


Acrobat logo Download manual as PDF

Splunk Enterprise version 7.0 is no longer supported as of October 23, 2019. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Search and monitor metrics

To analyze data in a metrics index, use mstats, which is a reporting command. Using mstats you can apply metric aggregations to isolate and correlate problems from different data sources. See mstats in the Search Reference manual.

To enumerate metric names, dimensions, and values, use mcatalog, which is an internal search command. See mcatalog in the Search Reference manual.

Other search commands do not work with a metrics index.

Note the following differences:

  • You cannot search metrics data for individual metric events.
  • You cannot use automatic lookups with metrics data. This is because automatic lookups are applied to individual events, whereas metrics are analyzed as an aggregate.
  • You cannot perform search-time extractions.
  • You can enrich metrics with the equivalent of custom indexed fields, which are treated as dimensions.
  • You can use reserved fields such as "source", "sourcetype", or "host" as dimensions. However, when extracted dimension names are reserved names, the name is prefixed with "extracted_" to avoid name collision. For example, if a dimension name is "host", search for "extracted_host" to find it.
  • Dimensions that start with underscore ( _ ) are not indexed, so they are not searchable.

Search examples

To list counts of metric names over 10-second intervals:

| mstats count where metric_name=* span=10s BY metric_name

To perform a simple count of a dimension:

| mstats count where index=mymetricsdata metric_name=aws.ec2.CPUUtilization

To calculate an average value of measurements for every 30-second interval:

| mstats avg(_value) WHERE index=mymetricdata AND metric_name=aws.ec2.CPUUtilization span=30s

You can also display results in a chart. The following example uses a wildcard search and group by:

| mstats avg(_value) prestats=t WHERE metric_name="cpu.*" span=1m by metric_name | timechart avg(_value) as "Avg" span=1m by metric_name

This type of search can be used to stack different CPU metrics that add up to 100%.

This screen image shows a search that displays metrics results in a chart

This search shows an example of using an EVAL statement:

| mstats avg(_value) as "Avg" WHERE metric_name="memory.free.value" span=5s | eval mem_gb = Avg / 1024 / 1024 / 1024 | timechart max("mem_gb") span=5s

To list all metric names in all metrics indexes:

| mcatalog values(metric_name)

To list all dimensions in all metrics indexes:

| mcatalog values(_dims)

Use the REST API to list metrics data

You can also use the Metrics Catalog REST API endpoints to enumerate metrics data:

  • Use the GET /services/catalog/metricstore/metrics endpoint to list metric names.
  • Use the GET /services/catalog/metricstore/dimensions endpoint to list dimension names.
  • Use the GET /services/catalog/metricstore/dimensions/{dimension-name}/values endpoint to list values for given dimensions.

You can also use filters with these endpoints to limit results by index, dimension, and dimension values.

See Metrics Catalog endpoint descriptions in the REST API Reference Manual.

Last modified on 28 March, 2018
Get metrics in from other sources
Metrics indexing performance

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters