Splunk® Enterprise

Metrics

Download manual as PDF

Download topic as PDF

Set up ingest-time log-to-metrics conversion in Splunk Web

You can set up ingest-time log-to-metrics conversion through Splunk Web. You might want log-to-metrics conversion to take place at ingest time if you want the Splunk platform to preserve the metric data points that result from the conversion in a specific metrics index.

Complete the following two tasks to set up log-to-metrics conversion at ingest time:

  • Create a source type in the Log to Metrics category.
  • Apply this source type to a log data input.

To use this functionality, your role must have the edit_metric_schema capability. If your role does not have it, and you need to set up ingest-time logs-to-metrics conversion through Splunk Web, contact your Splunk administrator.

Know your log data

Creation of a Log to Metrics source type requires you to have basic knowledge about the log data that you wish to convert into metric data points. You need to know the fields in your log data and the categories that those fields fit into.

Field category Description
Measurement A field whose numeric values become unique metric data points.
Dimension A field that provides additional metadata for metric data points. The Splunk platform counts as dimensions any fields it extracts from a log event that you have not already identified as measurements or blacklist fields. All metric data points generated from an event share the dimension field-value pairs in that event.
Blacklisted field A field in a log event that does not appear in the metric data points generated from that event. High-cardinality fields that are unimportant for the purposes of metric data point collection are good candidates for field blacklisting.

For example, say you have an event with a timestamp and the following five fields: max_kb, min_kb, server_model, group, and division. If you identify max_kb and min_kb as measurements, and you identify group and division as blacklist fields, the Splunk platform will generate two metric data points, one for each of the measurement fields. The metric data points will both share server_model as a dimension field.

Create a Log to Metrics source type

You can create a source type in the Log to Metrics category with the Source Types listing page in Settings.

Prerequisites

Steps

  1. Select Settings > Source types to open the Source Types listing page.
  2. Click New Source Type to open the Create Source Type dialog.
  3. Enter a Name for your new source type.
  4. (Optional) Enter a source type Description for your new source type. Select a different Destination app if necessary.
  5. Select Category > Log to Metrics.
  6. Select an appropriate Indexed Extractions value for your data.

    For example, if you are working with structured CSV- or JSON-formatted data, select csv or json, as appropriate. Use field_extraction if your data is technically unstructured but its events are strings of field-value pairs.
  7. (Optional) Change the settings on the Event Breaks, Timestamp, and Advanced tabs as necessary for your log data.
  8. Click on the Metrics tab to reveal the Log to Metrics source type settings.
    Text box label Optional? Description
    Measures No Enter one or more comma-separated names of numeric measurement fields.
    Blacklist Yes Enter one or more comma-separated names of dimension fields that you want to blacklist from the metric data points generated from the log events associated with this source type. You might want to blacklist high-cardinality dimension fields that are unnecessary for your metric collection.
  9. Click Save.

Apply a Log to Metrics source type to the data from an uploaded file or directory

After you create a source type in the Log to Metrics category, you can use the Set Source Type step of the Add Data workflow to apply the source type to data inputs that specify a single file as a source of data. When you set Log to Metric category source types to such inputs, a Metrics drop-down tab appears in the left pane of the Set Source Type page. Use this tab to enter or update lists of measures and blacklist dimensions for the source type.

The Add Data workflow is documented in full detail in Getting Data In.

Prerequisites

Steps

  1. Follow the Add Data workflow for uploading or monitoring a file or directory until you get to the Select Source Type page.
  2. On the Select Source Type page, select Source type > Log to Metrics and choose an appropriate source type from the list.
    When you select a Log to Metrics source type, the right-hand preview panel does not populate with a preview of the metrics data. You can see a preview for other source types.
  3. (Optional) Open the Event Breaks, Timestamp, and Advanced drop-down tabs and update their settings as necessary for your data input.
  4. (Optional) Open the Metrics drop-down tab to enter or update field lists in the Measures and Blacklist text boxes. Measures requires at least one field.
    Text box label Description
    Measures Review the list of comma-separated names of numeric measurement fields in this text box and update it if necessary. A unique metric data point is created for each measurement field-value pair in a log event associated with this source type.
    Blacklist This text box can contain a comma-separated list of dimension fields that you want to blacklist from the metric data points generated from the log events associated with this source type. You might want to blacklist high-cardinality dimension fields that are unnecessary for your metric collection.
  5. Click Next to continue with the Add Data workflow for your data input.
PREVIOUS
Overview of log-to-metrics conversion
  NEXT
Set up ingest-time log to metrics conversion with configuration files

This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.3.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters