Calico Pods Fails to Start Resulting in Failure to Establish Communication Between ETCD Pods on Master Nodes

Problem

• ETCD pods on Master Nodes fails to establish communication as the network is not configured due to failure to download the Images for Calico from docker registry. Below messages are observed in /var/log/pf9/kubelet/kubelet.INFO.

kubelet.INFO

I0615 11:35:07.881385   19822 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
E0615 11:35:07.893525   19822 remote_image.go:113] PullImage "calico/cni:v3.14.1" from image service failed: rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: AuthorizedOnly
E0615 11:35:07.893588   19822 kuberuntime_image.go:50] Pull image "calico/cni:v3.14.1" failed: rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: AuthorizedOnly
E0615 11:35:07.893664   19822 kuberuntime_manager.go:807] init container start failed: ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: AuthorizedOnly.

Environment

• Platform9 Managed Kubernetes - All Version

• HTTP_Proxy

Cause

• The HTTP Proxy defined by the customer infrastructure is blocking communication with docker registry. This can be confirmed by running the below command.

Sample Output

$ sudo docker pull calico/cni:v3.14.1
Error response from daemon: Get https://registry-1.docker.io/v2/: AuthorizedOnly

Resolution

1. Whitelist the docker registry in the http_proxy configuration in to enable communication.