Configure StatsD dimension extraction through REST API endpoints

If you use Splunk Cloud Platform or simply do not have Splunk filesystem access, you can configure dimension extraction for unsupported StatsD formats by making manual calls to REST API endpoints. You may need to configure dimension extraction if your StatsD client embeds dimensions in the metric name.

Prerequisites

• Read the first part of Configure dimension extraction for unsupported StatsD formats to get an overview of why dimension extraction might be necessary.
• See the following topics in the REST API Reference Manual to better understand the Splunk REST API system and the REST endpoints discussed in this procedure.
  • Using the REST API reference
  • /data/transforms/statsdextractions
  • /admin/metrics-reload/_reload
• If you use Splunk Cloud Platform, read Access requirements and limitations for the Splunk Cloud REST API in Rest API Tutorials.

Steps

1. Define a custom source type for your StatsD metrics data by using the /services/saved/sourcetypes REST endpoint:

   https://<host>:<mPort>/services/saved/sourcetypes   \
   -d "name=<metrics_sourcetype_name>&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=<statsd_dim_stanza_name>&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
   

   • metrics_sourcetype_name: The name of your custom metrics source type.
   • statsd_dim_stanza_name: A list of transforms stanza names that specify how to extract dimensions. If only one stanza is used for the source type, and if the transforms stanza name is same as the metrics_sourcetype_name, this STATSD-DIM-TRANSFORMS setting can be omitted.

   For example, enter the following command:

   curl -k -u admin:changeme https://localhost:8089/services/saved/sourcetypes   \
   -d "name=statsd_custom&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=statsd-ex&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
   

2. Create one or more regular expressions to extract the dimensions from metric_name by using the /data/transforms/statsdextractions REST endpoint:

   https://<host>:<mPort>/services/data/transforms/statsdextractions \
   -d "name=<unique_transforms_stanza_name>&REGEX=<regular expression>&REMOVE_DIMS_FROM_METRIC_NAME=<Boolean>"
   

   • unique_transforms_stanza_name: A unique name for this stanza.
   • REGEX = <regular expression>: A regular expression that defines how to match and extract dimensions from StatsD metrics data. The Splunk platform supports a named capturing-group extraction format (?<dim1>group)(?<dim2>group)... to provide dimension names for the corresponding values that are extracted.
   • REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>: Specifies whether unmatched segments of the StatsD dotted name segment are used as the metric_name.

   When true, dimension values are be removed from the measurement and the unmatched portion becomes the metric_name. The default value is true.

   When false, extracted dimension values are included in the metric_name.

   For example, a metric measurement name is "x.y.z". The regular expression matches "y" and "z". When REMOVE_DIMS_FROM_METRIC_NAME is true, metric_name is "x". When false, metric_name is "x.y.z".

   For example, enter the following command:

   curl -k -u admin:changeme https://localhost:8089/services/data/transforms/statsdextractions \
   -d "name=statsd-ex&REGEX=\.(?<hostname>\S%2B?)\.(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})&REMOVE_DIMS_FROM_METRIC_NAME=true"
   

3. Reload the metrics processor to load the configuration changes by using the /admin/metrics-reload/_reload REST endpoint:

   https://<host>:<mPort>/services/admin/metrics-reload/_reload
   

   For example, enter the following command:

   curl -k -u admin:changeme \
   https://localhost:8089/services/admin/metrics-reload/_reload
   

4. Create a data input for this source type as described in Set up a data input for StatsD data, and select your custom source type.