"logging before flag.Parse" Errors Observed For metrics-server POD.

Problem

• The following errors are observed in metrics-server-nannycontainer logs of metrics-serverPOD.

Errors

# kubectl logs metrics-server-v0.5.0-5fd89488cd-dng75 -n kube-system metrics-server-nanny
ERROR: logging before flag.Parse: I1003 14:54:21.545085       1 pod_nanny.go:68] Invoked by [/pod_nanny --config-dir=/etc/config --cpu=40m --extra-cpu=0.5m --memory=40Mi --extra-memory=4Mi --threshold=5 --deployment=metrics-server-v0.5.0 --container=metrics-server --poll-period=30000 --estimator=exponential --minClusterSize=16 --use-metrics=true]
ERROR: logging before flag.Parse: I1003 14:54:21.545257       1 pod_nanny.go:69] Version: 1.8.14
ERROR: logging before flag.Parse: I1003 14:54:21.545318       1 pod_nanny.go:85] Watching namespace: kube-system, pod: metrics-server-v0.5.0-5fd89488cd-dng75, container: metrics-server.
ERROR: logging before flag.Parse: I1003 14:54:21.545352       1 pod_nanny.go:86] storage: MISSING, extra_storage: 0Gi
ERROR: logging before flag.Parse: I1003 14:54:21.645992       1 pod_nanny.go:116] cpu: 40m, extra_cpu: 0.5m, memory: 40Mi, extra_memory: 4Mi
ERROR: logging before flag.Parse: I1003 14:54:21.646061       1 pod_nanny.go:145] Resources: [{Base:{i:{value:40 scale:-3} d:{Dec:<nil>} s:40m Format:DecimalSI} ExtraPerNode:{i:{value:5 scale:-4} d:{Dec:<nil>} s: Format:DecimalSI} Name:cpu} {Base:{i:{value:41943040 scale:0} d:{Dec:<nil>} s: Format:BinarySI} ExtraPerNode:{i:{value:4194304 scale:0} d:{Dec:<nil>} s:4Mi Format:BinarySI} Name:memory}]

• No adverse effect observed on the monitoring stack or Platform9 Managed Kubernetes cluster.

Environment

• Platform9 Managed Kubernetes - All Versions
• Metric Server
• Add-on Resizer

Cause

• Add-on Resizer use Google/glog as logging library.

Add-On Resizer Code snippet

https://github.com/kubernetes/autoscaler/blob/addon-resizer-release-1/addon-resizer/nanny/main/pod_nanny.go#L45

package main

import (
"fmt"
"io/ioutil"
"os"
"time"
...
"k8s.io/apimachinery/pkg/api/resource"
"k8s.io/autoscaler/addon-resizer/nanny"

"github.com/golang/glog"   <-------------------------------
"k8s.io/apimachinery/pkg/runtime"
...
"path/filepath"
)

• A commit in glog seems to be responsible for generating those error messages but there are no adverse effects of these messages on metric server or cluster functionality.
• The compatible changes for this commit have not done in Kubernetes code, instead it is considered as a caveat.

Resolution

• Since Kubernetes considers this as a caveat and there are no adverse effect reported for these errors hence we can safely ignore them.