How To Enable Verbose Logging for Kubelet on a Node?

Problem

• Procedure to increase the log verbosity of pf9-kubelet service to a higher level.

Environment

• Platform9 Managed Kubernetes - All versions

Procedure

• Check the current verbosity level of process pf9-kubelet.

Check current verbosity level for pf9-kubelet service

# ps -ef | grep kubelet
root     26642     1  3 18:15 ?        00:00:04 /opt/pf9/pf9-kube/bin/kubelet --kubeconfig=/etc/pf9/kube.d/kubeconfigs/kubelet.yaml --enable-server --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --log-dir=/var/log/pf9/kubelet/ --logtostderr=false --config=/var/opt/pf9/kube/kubelet-config/bootstrap-config.yaml --register-schedulable=false --pod-infra-container-image=localhost:5100/pause:3.2 --dynamic-config-dir=/var/opt/pf9/kube/kubelet-config/dynamic-config --hostname-override=10.69.74.160 --v=2

• Edit systemd service file for pf9-kubelet i.e. /run/systemd/system/pf9-kubelet.service to increase logging verbosity to --v=8.

Changing pf9-kubelet service verbosity level

[root@worker1 ~]# vi /run/systemd/system/pf9-kubelet.service

[root@worker1 ~]# cat /run/systemd/system/pf9-kubelet.service | grep  "v="
         --kubeconfig=/etc/pf9/kube.d/kubeconfigs/kubelet.yaml         --enable-server         --network-plugin=cni         --cni-conf-dir=/etc/cni/net.d         --cni-bin-dir=/opt/cni/bin         --log-dir=/var/log/pf9/kubelet/         --logtostderr=false         --config=/var/opt/pf9/kube/kubelet-config/bootstrap-config.yaml         --register-schedulable=false         --pod-infra-container-image=localhost:5100/pause:3.2         --dynamic-config-dir=/var/opt/pf9/kube/kubelet-config/dynamic-config --hostname-override=10.128.146.217 --v=8

• Reload the systemd daemon and restart the pf9-kubelet service on the node post above changes.

Perform systemd service restart

[root@worker1 ~]# systemctl daemon-reload
[root@worker1 ~]# systemctl restart pf9-kubelet

• Once again check and verify the verbosity level of process pf9-kubelet.

Verify the updated verbosity level for pf9-kube service

# ps -ef | grep kubelet
root     26642     1  3 18:15 ?        00:00:04 /opt/pf9/pf9-kube/bin/kubelet --kubeconfig=/etc/pf9/kube.d/kubeconfigs/kubelet.yaml --enable-server --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --log-dir=/var/log/pf9/kubelet/ --logtostderr=false --config=/var/opt/pf9/kube/kubelet-config/bootstrap-config.yaml --register-schedulable=false --pod-infra-container-image=localhost:5100/pause:3.2 --dynamic-config-dir=/var/opt/pf9/kube/kubelet-config/dynamic-config --hostname-override=10.69.74.160 --v=8