Docs » Monitor services and hosts in Splunk Infrastructure Monitoring » Monitor Kubernetes

Monitor Kubernetes 🔗

Note

The following topic describes the new Kubernetes navigator. See Monitor Kubernetes (classic version) for documentation on the classic Kubernetes navigator.

You can monitor Kubernetes metrics with Splunk Observability Cloud. Splunk Observability Cloud uses the Splunk Distribution of OpenTelemetry Collector for Kubernetes to provide robust infrastructure monitoring capabilities. To learn more, see Get started with the Splunk Distribution of the OpenTelemetry Collector.

Prerequisites 🔗

Before you start monitoring any Kubernetes resources, do the following:

Kubernetes navigators 🔗

Note

The following sections show you components specific to the Kubernetes navigators. For information on components shared by all navigators, see Use navigators in Splunk Infrastructure Monitoring.

There are two Kubernetes navigators, Kubernetes nodes and Kubernetes workloads. On the Infrastructure landing page, you can see the summary cards for both navigators under the Kubernetes section.

The following table compares the two Kubernetes navigators.

Navigator

Description

Use this to

Kubernetes nodes

Provides a hierarchical view of your Kubernetes infrastructure

  • Get an overview of your entire Kubernetes infrastructure

  • Monitor the health of all or part of the Kubernetes infrastructure

  • Identify and diagnose an issue with some part of the Kubernetes infrastructure

Kubernetes workloads

Provides a view of Kubernetes workloads across all your infrastructure

Monitor Kubernetes workloads across your infrastructure, or a specific subset of workloads, such as those running in a particular namespace.

Kubernetes nodes navigator 🔗

Each Kubernetes service consists of the following elements:

  • Container: A lightweight package containing everything needed to run applications.

  • Pod: A group of one or more containers, with shared storage and network resources, and a specification for how to run the containers.

  • Node: A physical or a virtual machine that hosts pods and the necessary resources to run pods.

  • Cluster: A group of nodes for running containerized applications.

Diagram of Kubernetes component hierarchical relationship.

Monitor your entire Kubernetes infrastructure with an interactive hierarchical map. You can select elements in the map to drill down into them, or use the filter to explore your data. The level of detail shown on the map is dynamic and depends on the number of elements shown.

Containers, pods, and nodes are colored by health and status, as reported by Kubernetes:

  • Containers are colored by status: Ready, Not Ready, and Unknown

  • Pods are colored by phase: Running, Pending, Succeeded, Failed, and Unknown

  • Nodes are colored by condition: Node Ready, Memory Pressure, PID Pressure, Disk Pressure, Network Unavailable, and Out of Disk

Investigate instances in the hierarchical map 🔗

  • Breadcrumb navigation: Jump across levels and switch to different entities at any level using the breadcrumb navigation bar.

  • Hover: Get more information about an element, including status or phase, by hovering over that element.

  • Select and zoom: Drill down into an element and change the zoom level of the map, if applicable, by selecting the element. Details about the element display in the sidebar, in the About this pod panel.

  • Filter: Filter the map by any available metadata in your Kubernetes data, such as a namespace, a workload, or any other key-value pair. When you apply a filter, the map shows only nodes that match the filter and highlights matching pods and containers. You can still select the dimmed pods and containers to view details about them in the sidebar.

Kubernetes workloads navigator 🔗

A workload is an application running on Kubernetes. Your workload might be a single component or several that work together, but it always runs inside a set of pods on Kubernetes.

Instead of a hierarchical approach to your Kubernetes infrastructure, you can investigate workloads for a given Kubernetes namespace, and the pods where each workload is running on.

For more information, see Use navigators in Splunk Infrastructure Monitoring.

View services and hosts on which Kubernetes is running 🔗

Apart from monitoring your Kubernetes infrastructure, you can also track services and hosts where Kubernetes is running in the navigator sidebar for both the Kubernetes nodes and workloads navigators. When you select a host or service from the sidebar, you are switching to the navigator for that host or service instance.

Note

From a host navigator, you can also jump to a Kubernetes navigator, but only to the Kubernetes nodes navigator.

Navigating to the EC2 navigator from the Kubernetes nodes navigator, and then navigating back to the Kubernetes nodes navigator.

Analyzer 🔗

The Analyzer accessed through the K8s analyzer tab helps you troubleshoot Kubernetes problems at scale by highlighting Kubernetes objects that are in a bad state, such as nodes that are not ready. Then, the Analyzer produces theories about what those objects might have in common, such as that all of the objects are running the same workload or all objects are located in the same AWS region. Click on a finding in the Analyzer panel to filter the map.

The Analyzer panel displays suggested filters for the elements selected in the cluster map. Click links in the Analyzer panel to add filters to the cluster map and explore conditions across your entire Kubernetes environment.

The Analyzer uses AI-driven insights to examine patterns that nodes, pods, or containers could have in common. Trouble indicators are:

  • pods that are in pending status

  • pods that are in failed status

  • pods with unknown condition

  • containers with high restart counts

  • nodes not ready

  • nodes with unknown condition

  • nodes experiencing high CPU

  • nodes experiencing high memory

The Analyzer displays overrepresented metrics properties for known conditions, such as pods in pending status, pods in failed status, and so on. You can use properties that are highly correlated with these conditions to filter the cluster map. You can explore data about each of those elements in the navigator using context-sensitive dashboards. This enables you to identify the underlying patterns noticeable on the filtered map that might be correlated with Kubernetes issues. For example, if all failed pods are in certain types of clusters, Analyzer provides suggested paths to follow for troubleshooting such issues.

Next steps 🔗

If you’re also exporting logs from Kubernetes and want to learn about how to view logs in Observability Cloud, see Introduction to Splunk Log Observer.

You can also export and monitor data related to your Kubernetes clusters, as described in the following table.

Get data in

Monitor

Description

Connect to the cloud service provider your Kubernetes clusters run in, if any.

Instrument back-end applications to send spans to Splunk APM

Introduction to Splunk APM

Collect metrics and spans from applications running in Kubernetes clusters.