Convert event logs to metric data points

Metrics are often buried in unstructured or semi-structured log data. The Splunk platform can automatically convert log data to metrics data points and then insert that data into a metrics index that you specify. It can perform this conversion when your log data is ingested into your Splunk platform deployment, or when you run a search on the log data with the mcollect or meventcollect commands.

This functionality follows older features for the Splunk platform that enable the extraction of fields from events at ingest time and search time. When you set up a log-to-metrics conversion, you look at the field-value pairs that are pulled out of your unstructured events and identify the numeric fields that are captured as measurement fields. The Splunk platform generates a separate, unique metric data point for each measurement field in an event.

You can optionally identify extracted fields for the Splunk platform to blacklist so they do not appear in the metric data points.

Extracted fields in your events that you have not identified as measurements or blacklisted fields are added by the Splunk platform to metric data points as dimensions. All metric data points generated from the same event share the same dimension field-value pairs.

Certain log-to-metrics feature extensions, such as the ability to create log-to-metric configurations that automatically process numeric fields in events as metric measures, can only be managed through manual configuration file edits or REST API operations.

Log events

Here are two log events that contain metrics data:

_time	Event
08-05-2017 20:26:29.073 -0700	INFO Metrics - group=queue, name=aeq, max_size_kb=500, current_size_kb=300, current_size=53, largest_size=65, smallest_size=5
08-05-2017 20:26:29.075 -0700	INFO Metrics - group=queue, name=indexqueue, max_size_kb=500, current_size_kb=200, current_size=55, largest_size=85, smallest_size=0

The Splunk platform runs a process that extracts the field-value pairs from the events and displays the following table.

_time	group	name	max_size_kb	current_size_kb	current_size	largest_size	smallest_size
08-05-2017 20:26:29.073 -0700	queue	aeq	500	300	53	65	5
08-05-2017 20:26:29.075 -0700	queue	indexqueue	500	200	55	85	0

If you identify the measurement fields in these events as max_size_kb, current_size_kb, current_size, largest_size, and smallest_size, the Splunk platform generates a unique metric data point for each extraction of those fields. The following table has 10 metric data points, one for each measurement field-value pair in the two original log events.

_time	metric_name	_value	group	name
08-05-2017 20:26:29.073 -0700	max_size_kb	500	queue	aeq
08-05-2017 20:26:29.073 -0700	current_size_kb	300	queue	aeq
08-05-2017 20:26:29.073 -0700	current_size	53	queue	aeq
08-05-2017 20:26:29.073 -0700	largest_size	65	queue	aeq
08-05-2017 20:26:29.073 -0700	smallest_size	5	queue	aeq
08-05-2017 20:26:29.075 -0700	max_size_kb	500	queue	indexqueue
08-05-2017 20:26:29.075 -0700	current_size_kb	200	queue	indexqueue
08-05-2017 20:26:29.075 -0700	current_size	55	queue	indexqueue
08-05-2017 20:26:29.075 -0700	largest_size	85	queue	indexqueue
08-05-2017 20:26:29.075 -0700	smallest_size	0	queue	indexqueue

Anatomy of a log-to-metrics metric data point

Each metric data point contains a _time field, a metric_name field, and a _value field. Metric data points can have one or more dimension fields. Learn more about metric data points in Overview of metrics.

The following table explains how the log-to-metrics process derives the values of each metric data point field:

Metric field	Example value	Origin of value
_time	08-05-2017 20:26:29.075 -0700	Uses the _time value from the original event. If multiple metric data points are generated from a single event, they all share the same _time value.
metric_name	largest_size	Uses the name of the measurement field that provides the _value for the metric data point. In this case the measurement field was named largest_size.
_value	85	Uses the value of the measurement field that the metric data point is based on. In this case, the metric data point is based on largest_size=85.
dimension field	group=queue, name=indexqueue	Any fields in a log event besides _time and metric_name that are not otherwise identified as measurement fields or blacklisted fields become dimension fields. All metric data points generated from the same log event share the same timestamps and dimension field-value pairs.

Set up basic ingest-time log-to-metric conversions through Splunk Web

Use Splunk Web to set up ingest-time conversion of logs to metric data points when all of the events in the log being ingested share the same fields.

There are two stages to the Splunk Web process for setting up log-to-metrics conversion:

1. Create a new source type of the Log to Metrics category on the Source Types listing page in Settings.
2. Associate that Log to Metrics source type with an appropriate log data input when you create or edit the input.

For more information, see Set up ingest-time log-to-metrics conversion in Splunk Web.

Create sophisticated ingest-time log-to-metric conversions with props.conf and transforms.conf

Manually create configurations in transforms.conf and props.conf for ingest-time conversion of logs to metric data points when the events in the log being ingested have different sets of measurement fields. For example, you can design configurations that sort events by the values of a shared field and then apply specific log-to-metric conversion rules to each of those event groups.

For more information, see Set up ingest-time log-to-metrics conversion with configuration files.

Numeric fields that are never converted to metric measures

Certain numeric field names are reserved. The Splunk software cannot convert indexed fields with these names to metric measures. If you use these names for your indexed measure fields, you should arrange to have them renamed before they undergo log-to-metric processing. Such renaming will require changes to your transforms.conf configurations.

This is the list of reserved field names:

• _event_status
• _indextime
• _subsecond
• _value
• date_hour
• date_mday
• date_minute
• date_month
• date_second
• date_wday
• date_year
• date_zone
• linecount
• timeendpos
• timestartpos
• metric_timestamp
• punct
• time
• timestamp