Troubleshoot Network Explorer ๐
Network Explorer isnโt showing metrics as expected ๐
When Network Explorer experiences data issues, you might see the following:
Very limited or no TCP metrics and HTTP metrics.
No workloads, or only workloads for
containerd
andkubelet
.
Causes ๐
Your kubelet
and containerd
cgroup drivers might be not be the same value.
Solution ๐
Check the cgroup drivers of your kubelet
and containerd
to make sure that they match. Both of them have to be either cgroupfs
or systemd
. For more information, see the Kubernetes documentation on cgroup drivers.
You donโt see Kubernetes metadata in Network Explorer metrics ๐
Your Network Explorer metrics are not generated with Kubernetes metadata.
Causes ๐
Your Kubernetes collector is turned off.
The Kubernetes collector is turned off if you see the following:
k8sCollector: enabled: false
If your Kubernetes collector is turned on, you can determine the root cause based on the logs for the
k8s-watcher
andk8s-relay
containers in thek8s-collector
pod.Run the following command to find your
k8s-collector
pod name.kubectl get pods
Run the following commands to retrieve the logs for the containers.
kubectl logs <POD_NAME> -c k8s-watcher kubectl logs <POD_NAME> -c k8s-relay
This is an example error message from the logs. In this case, the service account token is not available.
Error: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
Note
On initial startup, the
k8s-watcher
tries to connect to thek8s-relay
. If thek8s-relay
has not yet come up, you might see the following set of error messages.Error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp [::1]:8712: connect: connection refused"
These messages are expected and you can ignore them.
Solution ๐
If your Kubernetes collector is turned off, you need to turn it on by setting
k8sCollector.enabled
totrue
in the Network Explorer values file.k8sCollector: enabled: true
For other errors, see the Kubernetes documentation for more information on configuring the service account for the pod to turn on communication with the API server.