Create metrics from your logs with log metricization rules 🔗
Customers with a Splunk Log Observer entitlement in Splunk Observability Cloud must transition from Log Observer to Log Observer Connect by December 2023. With Log Observer Connect, you can ingest more logs from a wider variety of data sources, enjoy a more advanced logs pipeline, and expand into security logging. See Splunk Log Observer transition to learn how.
Log metricization rules allow you to create a log-derived metric showing an aggregate count of logs grouped by the dimension of your choice. While Log Observer visual analysis allows you to dynamically view aggregate metrics in the context of your query, log metricization rules allow you to embed metrics from log data in charts, dashboards, and detectors. Log metricization rules enable you to see trends in your full logs data set without paying to index all of your logs data.
Only customers with a Splunk Log Observer entitlement in Splunk Observability Cloud can create log metricization rules. If you do not have a Log Observer entitlement and are using Splunk Log Observer Connect instead, see Introduction to Splunk Log Observer Connect to learn what you can do with the Splunk Enterprise integration.
Order of execution of logs pipeline rules 🔗
Logs pipeline rules execute in the following order:
Log processing rules
Log metricization rules
Infinite logging rules
Log Observer indexes your logs data only after executing all pipeline management rules. When you metricize then archive a set of logs, metricized logs count against your ingest capacity but not against your indexing capacity. Like any other metric, a metric derived from log metricization rules counts toward your metrics quota per your contract. For more information, see Sequence of logs pipeline rules.
All pipeline management rules, including log metricization rules, apply only to logs that are sent to Splunk Observability Cloud. You can’t apply log metricization rules, or any pipeline management rules, to logs viewed in Log Observer Connect because logs are not ingested into Observability Cloud. Log Observer Connect lets users view and analyze Splunk Platform logs but can’t transform them.
Create log metricization rules 🔗
There are two ways to create log metricization rules:
Create a log metricization rule from the logs pipeline 🔗
To create a new log metricization rule from scratch in the logs pipeline, follow these steps:
From the navigation menu, go to Data Configuration > Logs Pipeline Management.
Click New Metricization Rule.
Define a matching condition. Only matching logs will be included in the chart resulting from your metricization rule.
To configure a metric, perform a Log Observer aggregation query. Select a function, an aggregate, and a dimension for this query. You can choose from the following functions: Count, AVG, MAX, MIN, and SUM. The default function is Count. The default aggregate for Log Observer is All(*), and the default dimension is severity. Log Observer Connect has no default aggregation. To change the dimension of the aggregation, select another dimension in the Group by field. To See Group logs by fields using log aggregation for a thorough explanation of aggregation queries.
Next, select a target field by which you want to aggregate logs. For example, you can choose services as your target field, then group logs by status. Fields with “#”, such as amount, require a numerical value to aggregate logs.
Review your metric time series (MTS) summary to see how your metricization could affect your subscription usage. You can optionally select an ingest token to limit the MTS count.
Give your metric a name. The name defaults to the function and target fields.
You can optionally change the Metric Type to Gauge, Counter, or Cumulative counter.
Give your rule a name and description.
Review your configuration, then click Save. Your rule appears in the list of Metricization Rules on the Logs Pipeline Management page. Click the name of your rule to view a summary of the rule. To view the output of your rule, click view your new metric in a chart. This takes you to chart builder populated with your new metric. In less than 60 seconds, you will see metrics reported within the chart.
While still in chart builder, click Save As to save your new metric as a chart. You can then embed it on a new or existing dashboard.
Create a log metricization rule in the context of a Log Observer query 🔗
Often, you might notice the potential value of an existing query and decide to create a log metricization rule based on that query. You can quickly launch the creation of a new metricization rule from a Log Observer query.
To create a new log metricization rule in the context of an existing search query, follow these steps:
In the navigation menu, go to Log Observer.
Create a query that aggregates logs. See Group logs by fields using log aggregation to learn how.
In the Save menu, select Save as Metric. This takes you to the Configure Metric page in Logs Pipeline Management.
Go to step 3 in Create a log metricization rule from the logs pipeline and complete the instructions.
Log metricization rules limits 🔗
An organization can create a total of 128 log metricization rules.