Monitor Kubernetes 🔗
Note
The following topic describes the new Kubernetes navigator. See Monitor Kubernetes (classic version) for documentation on the classic Kubernetes navigator.
You can monitor Kubernetes metrics with Splunk Observability Cloud. Splunk Observability Cloud uses the Splunk Distribution of OpenTelemetry Collector for Kubernetes to provide robust infrastructure monitoring capabilities. To learn more, see Get started with the Splunk Distribution of the OpenTelemetry Collector.
Prerequisites 🔗
Before you start monitoring any Kubernetes resources, do the following:
Log in with your administrator credentials.
Kubernetes navigators 🔗
Note
The following sections show you components specific to the Kubernetes navigators. For information on components shared by all navigators, see Use navigators in Splunk Infrastructure Monitoring.
There are two Kubernetes navigators, Kubernetes nodes and Kubernetes workloads. On the Infrastructure landing page, you can see the summary cards for both navigators under the Kubernetes section.
The following table compares the two Kubernetes navigators.
Navigator
Description
Use this to
Kubernetes nodes
Provides a hierarchical view of your Kubernetes infrastructure
Get an overview of your entire Kubernetes infrastructure
Monitor the health of all or part of the Kubernetes infrastructure
Identify and diagnose an issue with some part of the Kubernetes infrastructure
Kubernetes workloads
Provides a view of Kubernetes workloads across all your infrastructure
Monitor Kubernetes workloads across your infrastructure, or a specific subset of workloads, such as those running in a particular namespace.
Kubernetes nodes navigator 🔗
Each Kubernetes service consists of the following elements:
Container: A lightweight package containing everything needed to run applications.
Pod: A group of one or more containers, with shared storage and network resources, and a specification for how to run the containers.
Node: A physical or a virtual machine that hosts pods and the necessary resources to run pods.
Cluster: A group of nodes for running containerized applications.
Monitor your entire Kubernetes infrastructure with an interactive hierarchical map. You can select elements in the map to drill down into them, or use the filter to explore your data. The level of detail shown on the map is dynamic and depends on the number of elements shown.
Containers, pods, and nodes are colored by health and status, as reported by Kubernetes:
Containers are colored by status:
Ready,Not Ready, andUnknownPods are colored by phase:
Running,Pending,Succeeded,Failed, andUnknownNodes are colored by condition:
Node Ready,Memory Pressure,PID Pressure,Disk Pressure,Network Unavailable, andOut of Disk
Investigate instances in the hierarchical map 🔗
Breadcrumb navigation: Jump across levels and switch to different entities at any level using the breadcrumb navigation bar.
Hover: Get more information about an element, including status or phase, by hovering over that element.
Select and zoom: Drill down into an element and change the zoom level of the map, if applicable, by selecting the element. Details about the element display in the sidebar, in the About this pod panel.
Filter: Filter the map by any available metadata in your Kubernetes data, such as a namespace, a workload, or any other key-value pair. When you apply a filter, the map shows only nodes that match the filter and highlights matching pods and containers. You can still select the dimmed pods and containers to view details about them in the sidebar.
View services and hosts on which Kubernetes is running 🔗
Apart from monitoring your Kubernetes infrastructure, you can also track services and hosts where Kubernetes is running in the navigator sidebar for both the Kubernetes nodes and workloads navigators. When you select a host or service from the sidebar, you are switching to the navigator for that host or service instance.
Note
From a host navigator, you can also jump to a Kubernetes navigator, but only to the Kubernetes nodes navigator.
Analyzer 🔗
The Analyzer accessed through the K8s analyzer tab helps you troubleshoot Kubernetes problems at scale by highlighting Kubernetes objects that are in a bad state, such as nodes that are not ready. Then, the Analyzer produces theories about what those objects might have in common, such as that all of the objects are running the same workload or all objects are located in the same AWS region. Click on a finding in the Analyzer panel to filter the map.
The Analyzer panel displays suggested filters for the elements selected in the cluster map. Click links in the Analyzer panel to add filters to the cluster map and explore conditions across your entire Kubernetes environment.
The Analyzer uses AI-driven insights to examine patterns that nodes, pods, or containers could have in common. Trouble indicators are:
pods that are in pending status
pods that are in failed status
pods with unknown condition
containers with high restart counts
nodes not ready
nodes with unknown condition
nodes experiencing high CPU
nodes experiencing high memory
The Analyzer displays overrepresented metrics properties for known conditions, such as pods in pending status, pods in failed status, and so on. You can use properties that are highly correlated with these conditions to filter the cluster map. You can explore data about each of those elements in the navigator using context-sensitive dashboards. This enables you to identify the underlying patterns noticeable on the filtered map that might be correlated with Kubernetes issues. For example, if all failed pods are in certain types of clusters, Analyzer provides suggested paths to follow for troubleshooting such issues.
Next steps 🔗
You can also export and monitor data related to your Kubernetes clusters, as described in the following table.
Get data in |
Monitor |
Description |
|---|---|---|
Connect to the cloud service provider your Kubernetes clusters run in, if any. |
||
Instrument back-end applications to send spans to Splunk APM |
Collect metrics and spans from applications running in Kubernetes clusters. |