Scenario: Kai configures a service level objective (SLO) based on a Synthetics check 🔗
Kai, a site reliability engineer at Buttercup Games, has created a Synthetics Browser test for the Emby service on the Buttercup Games website. Kai didn’t instrument the Emby service but wants to monitor the health of this service.
To measure and track the health metrics of the Emby service, Kai configures a custom metric SLO based on the Synthetics Browser test.
Use custom metric as service level indicator (SLI) 🔗
From the Detectors & SLOs page, Kai configures the SLI and sets up a target for their SLO. Kai follows these steps:
Kai wants to use a Synthetics metric as the system health indicators, so they select the Custom metric from the Metric type menu.
Kai enters following program into the SignalFlow editor:
G = data('synthetics.run.count', filter=filter('test', 'Monitoring Services - Emby check') and filter('success', 'true')) T = data('synthetics.run.count', filter=filter('test', 'Monitoring Services - Emby check'))
Kai defines variables
G
andT
as two streams ofsynthetics.run.count
metric time series (MTS) measuring the health of requests sent to the Emby service. To distinguish between the two data streams, Kai applies an additional filter on thesuccess
dimension in the definition forG
. This filter queries for a specific collection of MTS that track successful requests for the Emby service. In Kai’s SignalFlow program,G
is a data stream of good events andT
is a data stream of total events.Kai assigns
G
to the Good events (numerator) dropdown menu andT
to the Total events (denominator) dropdown menu.Kai enters the following fields to define a target for their SLO:
Field
Value
Description
Target (%)
99.9
Kai wants to measure the Emby service successful request rate against a 99.9% target.
Compliance window
Last 30 days
Kai wants to track this SLO over the past 30 days.
Kai subscribes to receive an alert whenever there is a breach event for the SLO target.
Summary 🔗
Kai set up an SLO and subscribed to breach event alerts that let them know when the Emby service on the Buttercup Games website fails to meet the request rate target. Kai managed to use custom metrics from an existing Synthetic Browser test to configure the SLO, despite not instrumenting the service themself.
Learn more 🔗
For more information about creating an SLO, see Measure and track your service health metrics with service level objectives (SLOs).
For more information about the Synthetics Browser test, see Browser tests for webpages.
For more information on SignalFlow, see Analyze data using SignalFlow in the Splunk Observability Cloud Developer Guide.