ImagePullBack Error on Airctl-Kplane And Airctl Namespaces

Problem

• The Platform9 component pods in the _airctl-kplane_ and _airctl_ namespaces in an ImagePullBackOff state.

Pod Output

kubectl get po -A
NAMESPACE               NAME                                                      READY   STATUS                  RESTARTS   AGE
airctl-kplane           ingress-nginx-controller-8ffb68576-v9st8                  1/1     Running                 0          7h26m
airctl-kplane           kplane-usermgr-65964f677d-dltcs                           0/1     ImagePullBackOff        0          7h26m
airctl-kplane           vault-renew-28344960-7m9zh                                0/1     ImagePullBackOff        0          5h33m
airctl                  clarity-6b9b58767d-fpxtz                                  0/1     ImagePullBackOff        0          3d8h
airctl                  docker-registry-d98f755bd-6fjv7                           2/2     Running                 0          3d8h
airctl                  forwarder-7df95899cd-825j9                                6/7     ImagePullBackOff        0          3d8h
airctl                  hagrid-768fcc548c-57qmv                                   2/3     ImagePullBackOff        0          3d8h

Events

Warning  Failed   38m (x61 over 4h46m)   kubelet  (combined from similar events): Failed to pull image "514845858982.dkr.ecr.us-west-1.amazonaws.com/pf9-vault:5.6.7-613": rpc error: code = Unknown desc = failed to pull and unpack image "514845858982.dkr.ecr.us-west-1.amazonaws.com/pf9-vault:5.6.7-613": failed to resolve reference "514845858982.dkr.ecr.us-west-1.amazonaws.com/pf9-vault:5.6.7-613": failed to do request: Head "https://514845858982.dkr.ecr.us-west-1.amazonaws.com/v2/pf9-vault/manifests/5.6.7-613": dial tcp: lookup 514845858982.dkr.ecr.us-west-1.amazonaws.com on 8.8.4.4:53: read udp 172.19.3.19:46588->8.8.4.4:53: i/o timeout

Environment

• Platform9 Edge Cloud

• LTS2 - v-5.6.7-2658688

Cause

• Utilising a custom private registry in place of the default registry.

Cluster_spec.yaml

privateRegistryType: custom
privateRegistryBase: <registry_Url>

• As of now,Platform9 do not support the use of a private registry for management plane components.

Resolution

• Manually load the management plane component images onto the nodes using the 'nerdctl' commands.

comamnds

1. nerdctl -n k8s.io load -i kubedu7-2658688.tar.gz

2. nerdctl -n k8s.io load docker-imgs-v-5.6.7-2658688.tar.gz

Additional Information

• RFE in place AIR-1259. To get the status of the RFE, please contact Platfrom9 Support