Extract dimensions for unsupported StatsD formats
Many StatsD clients embed dimension names in the metric name. For example, let's say your StatsD client uses the following line metric protocol format, which is not supported natively by the Splunk platform:
<dimension>.<metric_name>:<value>|<metric_type>
Here's an example of a metric returned using this unsupported format:
10.1.1.198.cpu.percent:75|g
The extracted metric fields should be:
metric_name=cpu.percent
_value=75
The extracted dimension should be:
ip=10.1.1.198
To create the correct results, you must edit Splunk configuration files or use the REST API to create a custom source type that specifies how to extract dimensions from this metrics data.
Configure dimension extraction by editing configuration files
- Define a custom source type for your StatsD metrics data.
- In a text editor, open the props.conf configuration file from the local directory for the location you want to use, such as the Search & Reporting app ($SPLUNK_HOME/etc/apps/search/local/) or from the system ($SPLUNK_HOME/etc/system/local). If a props.conf file does not exist in this location, create a text file and save it to that location.
- Append a stanza to the props.conf file as follows:
# props.conf [<metrics_sourcetype_name>] METRICS_PROTOCOL = statsd STATSD-DIM-TRANSFORMS = <statsd_dim_stanza_name1>,<statsd_dim_stanza_name2>...
- metrics_sourcetype_name: The name of your custom metrics source type.
- statsd_dim_stanza_name: A comma-separated list of transforms stanza names that specify how to extract dimensions. If only one stanza is used for the source type, and if the transforms stanza name is same as the metrics_sourcetype_name, this STATSD-DIM-TRANSFORMS setting can be omitted.
- Define one or more regular expressions to extract the dimensions from metric_name.
- In a text editor, open the transforms.conf configuration file from the local directory for the location you want to use, such as the Search & Reporting app ($SPLUNK_HOME/etc/apps/search/local/) or from the system ($SPLUNK_HOME/etc/system/local). If a transforms.conf file does not exist in this location, create a text file and save it to that location.
- Append a stanza for each regular expression as follows:
# transforms.conf [statsd-dims:<unique_transforms_stanza_name>] REGEX = <regular expression> REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>
- unique_transforms_stanza_name: A unique name for this stanza.
- REGEX = <regular expression>: A regular expression that defines how to match and extract dimensions from StatsD metrics data. The Splunk platform supports a named capturing-group extraction format
(?<dim1>group)(?<dim2>group)...
to provide dimension names for the corresponding values that are extracted. - REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>: Specifies whether unmatched segments of the StatsD dotted name segment are used as the metric_name.
When
true
, dimension values are removed from the measurement and the unmatched portion becomes the metric_name. The default value is true.When
false
, extracted dimension values are included in the metric_name.For example, a metric measurement name is "x.y.z". The regular expression matches "y" and "z". When REMOVE_DIMS_FROM_METRIC_NAME is
true
, metric_name is "x". Whenfalse
, metric_name is "x.y.z". - Create a data input for this source type as described in Set up a data input for StatsD data, and select your custom source type.
For more about editing these configuration files, see About configuration files, props.conf, and transforms.conf in the Admin Manual.
Examples of configuring dimension extraction
Let's say you have StatsD metrics data such as:
data=mem.percent.used.10.2.3.4.windows:33|g
You need to extract the "ipv4" and "os" dimensions.
If you defined two regular expressions, one for "ipv4" and one for "os", you would append the following stanzas to your configuration files:
# props.conf.example [my_custom_metrics_sourcetype] METRICS_PROTOCOL = statsd STATSD-DIM-TRANSFORMS = regex_stanza1, regex_stanza2
# transforms.conf.example [statsd-dims:regex_stanza1] REGEX = (?<ipv4>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}) REMOVE_DIMS_FROM_METRIC_NAME = true [statsd-dims:regex_stanza2] REGEX = \S+\.(?<os>\w+): REMOVE_DIMS_FROM_METRIC_NAME = true
Now let's say you can accomplish this same extraction using a single regular expression. In this case, you would append the following stanzas to your configuration files:
# props.conf.example [my_custom_metrics_sourcetype] METRICS_PROTOCOL = statsd
# transforms.conf.example [statsd-dims:my_custom_metrics_sourcetype] REGEX = (?<ipv4>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\.(?<os>\w+): REMOVE_DIMS_FROM_METRIC_NAME = true
Notice that the STATSD-DIM-TRANSFORMS setting in the props.conf configuration file is not needed when only a single regular expression is used for a source type.
Configure dimension extraction for StatsD by using the REST API
- Define a custom source type for your StatsD metrics data by using the /services/saved/sourcetypes REST endpoint:
https://<host>:<mPort>/services/saved/sourcetypes \ -d "name=<metrics_sourcetype_name>&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=<statsd_dim_stanza_name>&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
- metrics_sourcetype_name: The name of your custom metrics source type.
- statsd_dim_stanza_name: A list of transforms stanza names that specify how to extract dimensions. If only one stanza is used for the source type, and if the transforms stanza name is same as the metrics_sourcetype_name, this STATSD-DIM-TRANSFORMS setting can be omitted.
For example, enter the following command:
curl -k -u admin:changeme https://localhost:8089/services/saved/sourcetypes \ -d "name=statsd_custom&METRICS_PROTOCOL=statsd&STATSD-DIM-TRANSFORMS=statsd-ex&SHOULD_LINEMERGE=false&ANNOTATE_PUNCT=false&ADD_EXTRA_TIME_FIELDS=false&DATETIME_CONFIG=CURRENT&pulldown_type=true&category=Metrics"
- Create one or more regular expressions to extract the dimensions from metric_name by using the /data/transforms/statsdextractions REST endpoint:
https://<host>:<mPort>/services/data/transforms/statsdextractions \ -d "name=<unique_transforms_stanza_name>®EX=<regular expression>&REMOVE_DIMS_FROM_METRIC_NAME=<Boolean>"
- unique_transforms_stanza_name: A unique name for this stanza.
- REGEX = <regular expression>: A regular expression that defines how to match and extract dimensions from StatsD metrics data. The Splunk platform supports a named capturing-group extraction format
(?<dim1>group)(?<dim2>group)...
to provide dimension names for the corresponding values that are extracted. - REMOVE_DIMS_FROM_METRIC_NAME = <Boolean>: Specifies whether unmatched segments of the StatsD dotted name segment are used as the metric_name.
When
true
, dimension values are be removed from the measurement and the unmatched portion becomes the metric_name. The default value is true.When
false
, extracted dimension values are included in the metric_name.For example, a metric measurement name is "x.y.z". The regular expression matches "y" and "z". When REMOVE_DIMS_FROM_METRIC_NAME is
true
, metric_name is "x". Whenfalse
, metric_name is "x.y.z".For example, enter the following command:
curl -k -u admin:changeme https://localhost:8089/services/data/transforms/statsdextractions \ -d "name=statsd-ex®EX=\.(?<hostname>\S%2B?)\.(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})&REMOVE_DIMS_FROM_METRIC_NAME=true"
- Reload the metrics processor to load the configuration changes by using the /admin/metrics-reload/_reload REST endpoint:
https://<host>:<mPort>/services/admin/metrics-reload/_reload
For example, enter the following command:
curl -k -u admin:changeme \ https://localhost:8089/services/admin/metrics-reload/_reload
- Create a data input for this source type as described in Set up a data input for StatsD data, and select your custom source type.
For more about using the Splunk REST API, see Using the REST API reference, /data/transforms/statsdextractions, and /admin/metrics-reload/_reload in the REST API Reference Manual.
Get metrics in from StatsD | Get metrics in from collectd |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9
Feedback submitted, thanks!