Content Pack for Splunk Observability Cloud

Content Pack for Splunk Observability Cloud

This documentation does not apply to the most recent version of Content Pack for Splunk Observability Cloud. For documentation on the most recent version, go to the latest release.

KPI reference for the Content Pack for Splunk Observability Cloud

The Content Pack for Splunk Observability Cloud contains Applications Performance Monitoring services, Infrastructure Monitoring services, Real User Monitoring, and Synthetic Checks services. For each service, there are KPIs used to help you monitor the health of your applications. All parent and child services report up to the overall Splunk Observability Cloud service at the top level.

Applications Performance Monitoring services and KPIs

There are 4 Application Performance Monitoring services. This includes a service for Application Error Rate, Application Rate (Throughput), and Application Duration. Each of these roll up to a top-level Application Performance Monitoring service. For each of these services, there are KPIs used to help you monitor the health of your applications.

Service KPI Description
Application Duration SplunkAPM Duration Shows the combined application response time (duration) metrics from all ingested APM vendor metrics.


Thresholds: Low = 0-10 ms, Medium = 10-15 ms, High > 15 ms

Application Error Rate Splunk APM Error Rate Shows the combined percentage of requests with errors over total requests for the application from all Splunk APM Services reported metrics.


Thresholds: Normal <= 0.01 < Low < 25 < Medium < 50 < High < 90 < Critical

Application Rate (Throughput) SplunkAPM Rate (Throughput) Shows the combined application requests per second (Rate) from all ingested Splunk APM service metrics.


Thresholds: No predefined threshold. For steps to configure KPI thresholds, see Configure KPI thresholds in ITSI in the Service Insights manual.

Infrastructure Monitoring services and KPIs

There are 13 Infrastructure Monitoring services. This includes a service per entity type – OS Hosts, GCP Compute Engine, GCP Cloud Functions, Azure Functions. Each of these roll up to a corresponding parent service, which roll up to a top-level Infrastructure Monitoring service. For each of these services, there are KPIs used to help you monitor the health of your applications.

AWS services and KPIs

Service KPI Description
AWS EC2 AWS EC2 CPU Ultilization The percentage of allocated EC2 compute units that are currently in use on the instance.


Thresholds: High = 90, Medium = 80, Low = 50

AWS EC2 Disk Read Bytes The number of bytes read from all instance store volumes available to the instance.
AWS EC2 Disk Read Operations The completed read operations from all instance store volumes available to the instance in a specified period of time.
AWS EC2 Disk Write Bytes The completed write operations to all instance store volumes available to the instance in a specified period of time.
AWS EC2 Disk Write Operations The completed write operations to all instance store volumes available to the instance in a specified period of time.
AWS EC2 Network In The number of bytes received by the instance on all network interfaces. This metric identifies the volume of incoming network traffic to a single instance.


Thresholds: High = 0, Normal = 10

AWS EC2 Network Out The number of bytes sent out by the instance on all network interfaces. This metric identifies the volume of outgoing network traffic from a single instance.


Thresholds: High = 0, Normal = 10

AWS EC2 Network Packets In The number of packets received by the instance on all network interfaces. This metric identifies the volume of incoming traffic in terms of the number of packets on a single instance.


Thresholds: High = 0, Normal = 10

AWS EC2 Network Packets Out The number of packets sent out by the instance on all network interfaces. This metric identifies the volume of outgoing traffic in terms of the number of packets on a single instance.


Thresholds: High = 0, Normal = 10

AWS Lambda AWS Lambda Duration The amount of time that your function code spends processing an event. The billed duration for an invocation is the value of duration rounded up to the nearest millisecond.


Thresholds: High = 800,000, Medium = 400,000, Low = 200,000

AWS Lambda Errors The number of invocations that result in a function error. Function errors include exceptions thrown by your code and exceptions thrown by the Lambda runtime.


Thresholds: High = 1, Normal = 0

AWS Lambda Invocations The number of times your function code is executed, including successful executions and executions that result in a function error. Invocations aren't recorded if the invocation request is throttled or otherwise resulted in an invocation error.
AWS Lambda Throttles The number of invocation requests that are throttled. When all function instances are processing requests and no concurrency is available to scale up, Lambda rejects additional requests with TooManyRequestsException. Throttled requests and other invocation errors don't

Azure services and KPIs

Service KPI Description
Azure Functions Azure Functions 5xx Errors The count of 5xx errors from the Azure Function handler.


Thresholds: High = 1, Normal = 0

Azure Functions Bytes Received The rate at which the function process is receiving bytes to I/O operations.


Thresholds: High = 0, Normal = 100

Azure Functions Bytes Sent The rate at which the function process is sending bytes to I/O operations.


Thresholds: High = 0, Normal = 100

Azure Functions Executions The number of times your function has executed. This value correlates to the number of times a function runs in your app.
Azure Functions Memory The current amount of memory used by an Azure function.
Azure Functions Usage Cost The function cost in terms of function execution units which are a combination of execution time and your memory usage.
Azure VM Azure VM CPU Percentage The percentage of allocated compute units that are currently in use by the virtual machine.


Thresholds: High = 90, Medium = 80, Low = 50

Azure VM Disk Read Bytes The bytes read from disk during monitoring period.
Azure VM Disk Write Bytes The bytes written to disk during monitoring period.
Azure VM Network In The number of bytes received on all network interfaces by the virtual machine.


Thresholds: High = 0, Normal = 10

Azure VM Network Out The number of bytes out on all network interfaces by the virtual machine.


Thresholds: High = 0, Normal = 10

GCP services and KPIs

Service KPI Description
GCP Cloud Functions GCP Cloud Functions Active The number of active function instances.
GCP Cloud Functions Execution Time The distribution of functions execution times in nanoseconds.


Thresholds: High = 800,000, Medium = 400,000, Low = 200,000

GCP Cloud Functions Executions The count of function executions.
GCP Cloud Functions Memory The distribution of maximum functions' memory usage during execution, in bytes.
GCP Cloud Functions Network Egress The outgoing network traffic of function, in bytes.
GCP Compute Engine GCP Compute Engine CPU Utilization The fractional utilization of allocated CPU on this instance.


Thresholds: High = 90, Medium = 80, Low = 50

GCP Compute Engine Network Bytes In The count of bytes received from the network.


Thresholds: High = 0, Normal = 10

GCP Compute Engine Network Bytes Out The count of bytes sent over the network.


Thresholds: High = 0, Normal = 10

GCP Compute Engine Disk Read Bytes The count of bytes read on disk.
GCP Compute Engine Disk Write Bytes The count of bytes written on disk.

My Data Center Hosts services and KPIs

Service KPI Description
OS Hosts CPU Utilization CPU Utilization by %


Thresholds: High = 90, Medium = 80, Low = 50

Memory Free Memory free in megabytes.
Disk Write IOps Number of write IOps by a given instance.
Disk Read IOps Number of read IOps by a given instance.
Network Rx Network octets received.
Network Tx Network octets sent.
Kubernetes Pods CPU Utilization Pod CPU Utilization by %


Thresholds:High = 90, Medium = 80, Low = 50

Memory Utilization Pod Memory Utilization by %
Network Rx Errors Pod Network Receive Errors
Network Tx Errors Pod Network Transmit Errors
Docker Containers CPU Utilization


Thresholds:High = 90, Medium = 80, Low = 50

Disk Read Disk Read
Disk Write Disk Write
Memory Free Memory free in bytes

Splunk RUM services and KPIs

There are 6 Splunk Real User Monitoring (RUM)-related services. This includes two services each for RUM Browser, RUM Mobile, and RUM Synthetics.

Service KPI Description
RUM Browser Metrics Front-end requests Front-end requests by browser.
Front-end errors Front-end errors by browser.
Document load latency (P75) Document load latency in ms by browser (P75).
Endpoint Requests Browser Endpoint Requests by browser.

Endpoint Latency (P75)

Browser Endpoint Latency in ms by browser (P75).
RUM Browser Web Vitals Largest Contentful Paint in ms (P75) Largest Contentful Paint in ms by browser (P75).
Cumulative Layout Shift (P75) Cumulative Layout Shift by browser (P75).
First Input Delay in ms (P75) First Input Delay in ms by browser (P75).
RUM Mobile Metrics Front-end requests Front-end requests by app
Document load latency (P75) Document load latency in ms by app (P75).
Endpoint Requests Mobile Endpoint Requests by App.
Endpoint Latency (P75) Mobile Endpoint Latency in ms by App (P75).
RUM Mobile App Metrics App crash count Count of app crashes by app
App error Count of app errors by app
App Cold Start in ms (P75) App Cold Start times in ms (P75).
App Cold Starts Count of app cold starts.
RUM Synthetic Metrics Front-end requests Front-end requests by Splunk Synthetics.
Front-end errors Splunk Synthetics Front-end errors by browser.
Document load latency (P75) Splunk Synthetics Document load latency in ms by browser (P75).
Endpoint Requests Splunk Synthetics Endpoint Requests by browser.
Endpoint Latency (P75) Splunk Synthetics Endpoint Latency in ms by browser (P75).
RUM Synthetic Web Vitals Largest Contentful Paint in ms (P75) Splunk Synthetics Largest Contentful Paint in ms by browser (P75).
Cumulative Layout Shift (P75) Splunk Synthetics Cumulative Layout Shift by browser (P75).
First Input Delay in ms (P75) Splunk Synthetics First Input Delay in ms by browser (P75)

Synthetic Checks services and KPIs

There are 7 Synthetic Checks services. This includes a service per entity type or synthetic check, as well as a top-level synthetic check. For each of these services, there are KPIs used to help you monitor the health of your web applications. There is also an example service that you can use to create a custom configuration.

Note, these KPIs don't have predefined thresholds configured. For steps to configure KPI thresholds, see Configure KPI thresholds in ITSI in the Service Insights manual.

Service KPI Description
API Checks Failure Indicates the endpoint failed to respond.
Response Code The response code that is returned by the API endpoint.
Response Time The elapsed time for the response by the API endpoint.
Benchmark Checks Number of Competitors The number of competitors the page is checked against.
Rank The rank against all competitors.
Score A generated score from 0-100 that relates performance relative to the competition. It's calculated based on performance across various metrics. Rank is based on this score. Higher is better.
Content Checks Broken Links The count of links returning in HTTP error codes in the 400 range.
Links Checked The count of links checked on the webpage.
Warning Links The count of links returning HTTP codes in the 300 range.
HTTP Checks Days Since Last Error The count of days since the last error.
Failure Indicates the webpage failed to respond.
Response Code The response code that is returned by the web application.
Response Time The elapsed time for the response by the web application.
Browser Checks Failure Indicates the webpage failed to respond when accessed using the Chrome web browser.
Response Code The response code that is returned by the webpage when accessed using the Chrome web browser.
Response Time The elapsed time for the response by the webpage when accessed using the Chrome web browser.
Last modified on 29 November, 2022
Troubleshoot the Content Pack for Splunk Observability Cloud   Entity search reference for the Content Pack for Splunk Observability Cloud

This documentation applies to the following versions of Content Pack for Splunk Observability Cloud: 2.0.0


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters