Splunk® Enterprise

Metrics

Splunk Enterprise version 7.2 is no longer supported as of April 30, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Extract dimensions for unsupported StatsD formats

Many StatsD clients embed dimension names in the metric name. For example, let's say your StatsD client uses the following line metric protocol format, which is not supported natively by the Splunk platform:

<dimension>.<metric_name>:<value>|<metric_type>

Here's an example of a metric returned using this unsupported format:

10.1.1.198.cpu.percent:75|g

The extracted metric fields should be:

metric_name=cpu.percent
_value=75

The extracted dimension should be:

ip=10.1.1.198

To create the correct results, you must edit Splunk configuration files or use the REST API to create a custom source type that specifies how to extract dimensions from this metrics data.

Configure dimension extraction by editing configuration files

  1. Define a custom source type for your StatsD metrics data.
    1. In a text editor, open the props.conf configuration file from the local directory for the location you want to use, such as the Search & Reporting app ($SPLUNK_HOME/etc/apps/search/local/) or from the system ($SPLUNK_HOME/etc/system/local). If a props.conf file does not exist in this location, create a text file and save it to that location.
    2. Append a stanza to the props.conf file as follows: 
      # props.conf

[<metrics_sourcetype_name>]
METRICS_PROTOCOL = statsd
STATSD-DIM-TRANSFORMS = <statsd_dim_stanza_name1>,<statsd_dim_stanza_name2>...
      • metrics_sourcetype_name: The name of your custom metrics source type.
      • statsd_dim_stanza_name: A comma-separated list of transforms stanza names that specify how to extract dimensions. If only one stanza is used for the source type, and if the transforms stanza name is same as the metrics_sourcetype_name, this STATSD-DIM-TRANSFORMS setting can be omitted.
  2. Define one or more regular expressions to extract the dimensions from metric_name.
    1. In a text editor, open the transforms.conf configuration file from the local directory for the location you want to use, such as the Search & Reporting app ($SPLUNK_HOME/etc/apps/search/local/) or from the system ($SPLUNK_HOME/etc/system/local). If a transforms.conf file does not exist in this location, create a text file and save it to that location.
    2. Append a stanza for each regular expression as follows: 
      # transforms.conf

[statsd-dims:<unique_transforms_stanza_name>]
REGEX = <regular expression>
REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>
      • unique_transforms_stanza_name: A unique name for this stanza.
      • REGEX = <regular expression>: A regular expression that defines how to match and extract dimensions from StatsD metrics data. The Splunk platform supports a named capturing-group extraction format (?<dim1>group)(?<dim2>group)... to provide dimension names for the corresponding values that are extracted.
      • REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>: Specifies whether unmatched segments of the StatsD dotted name segment are used as the metric_name.

        • When true, dimension values are removed from the measurement and the unmatched portion becomes the metric_name. The default value is true.

        When false, extracted dimension values are included in the metric_name.

        For example, a metric measurement name is "x.y.z". The regular expression matches "y" and "z". When REMOVE_DIMS_FROM_METRIC_NAME is true, metric_name is "x". When false, metric_name is "x.y.z".

  3. Create a data input for this source type as described in Set up a data input for StatsD data, and select your custom source type.

For more about editing these configuration files, see About configuration files, props.conf, and transforms.conf in the Admin Manual.

Examples of configuring dimension extraction

Let's say you have StatsD metrics data such as:

data=mem.percent.used.10.2.3.4.windows:33|g

You need to extract the "ipv4" and "os" dimensions.

If you defined two regular expressions, one for "ipv4" and one for "os", you would append the following stanzas to your configuration files: 

# props.conf.example

[my_custom_metrics_sourcetype]
METRICS_PROTOCOL = statsd
STATSD-DIM-TRANSFORMS = regex_stanza1, regex_stanza2

# transforms.conf.example

[statsd-dims:regex_stanza1]
REGEX = (?<ipv4>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})
REMOVE_DIMS_FROM_METRIC_NAME = true
[statsd-dims:regex_stanza2]
REGEX = \S+\.(?<os>\w+):
REMOVE_DIMS_FROM_METRIC_NAME = true

Now let's say you can accomplish this same extraction using a single regular expression. In this case, you would append the following stanzas to your configuration files: 

# props.conf.example

[my_custom_metrics_sourcetype]
METRICS_PROTOCOL = statsd

# transforms.conf.example

[statsd-dims:my_custom_metrics_sourcetype]
REGEX = (?<ipv4>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\.(?<os>\w+):
REMOVE_DIMS_FROM_METRIC_NAME = true

Notice that the STATSD-DIM-TRANSFORMS setting in the props.conf configuration file is not needed when only a single regular expression is used for a source type.

Configure dimension extraction for StatsD by using the REST API

  1. Define a custom source type for your StatsD metrics data by using the /services/saved/sourcetypes REST endpoint: 
    https://<host>:<mPort>/services/saved/sourcetypes   \
-d "name=<metrics_sourcetype_name>&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=<statsd_dim_stanza_name>&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
    • metrics_sourcetype_name: The name of your custom metrics source type.
    • statsd_dim_stanza_name: A list of transforms stanza names that specify how to extract dimensions. If only one stanza is used for the source type, and if the transforms stanza name is same as the metrics_sourcetype_name, this STATSD-DIM-TRANSFORMS setting can be omitted.

    For example, enter the following command: 

    curl -k -u admin:changeme https://localhost:8089/services/saved/sourcetypes   \
-d "name=statsd_custom&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=statsd-ex&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
  2. Create one or more regular expressions to extract the dimensions from metric_name by using the /data/transforms/statsdextractions REST endpoint: 
    https://<host>:<mPort>/services/data/transforms/statsdextractions \
-d "name=<unique_transforms_stanza_name>&REGEX=<regular expression>&REMOVE_DIMS_FROM_METRIC_NAME=<Boolean>"
    • unique_transforms_stanza_name: A unique name for this stanza.
    • REGEX = <regular expression>: A regular expression that defines how to match and extract dimensions from StatsD metrics data. The Splunk platform supports a named capturing-group extraction format (?<dim1>group)(?<dim2>group)... to provide dimension names for the corresponding values that are extracted.
    • REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>: Specifies whether unmatched segments of the StatsD dotted name segment are used as the metric_name.

      • When true, dimension values are be removed from the measurement and the unmatched portion becomes the metric_name. The default value is true.

      When false, extracted dimension values are included in the metric_name.

      For example, a metric measurement name is "x.y.z". The regular expression matches "y" and "z". When REMOVE_DIMS_FROM_METRIC_NAME is true, metric_name is "x". When false, metric_name is "x.y.z".

    For example, enter the following command: 

    curl -k -u admin:changeme https://localhost:8089/services/data/transforms/statsdextractions \
-d "name=statsd-ex&REGEX=\.(?<hostname>\S%2B?)\.(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})&REMOVE_DIMS_FROM_METRIC_NAME=true"
  3. Reload the metrics processor to load the configuration changes by using the /admin/metrics-reload/_reload REST endpoint: 
    https://<host>:<mPort>/services/admin/metrics-reload/_reload

    For example, enter the following command: 

    curl -k -u admin:changeme \
https://localhost:8089/services/admin/metrics-reload/_reload
  4. Create a data input for this source type as described in Set up a data input for StatsD data, and select your custom source type.

For more about using the Splunk REST API, see Using the REST API reference, /data/transforms/statsdextractions, and /admin/metrics-reload/_reload in the REST API Reference Manual.

Last modified on 22 October, 2019
Get metrics in from StatsD   Get metrics in from collectd

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters