MetalLB AddOn Failing repository does not exist or may require 'docker login'

Problem

While trying to install or upgrade MetalLB Addon from Management Plane the docker logs shows following error:

logs

Error response from daemon: pull access denied for metallb/controller, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Environment

• Platform9 Managed Kubernetes - v5.4 and Higher

• MetalLB AddOn

Cause

MetalLB images that were earlier hosted on Docker have now moved to Quay officially. The old deployments are trying to reach the docker image repository which is causing this failure.

Resolution

Upgrade the Kubernetes cluster to any of the below list or higher version, that contains the fix with correct MetalLB repository

pf9-kube Versions

v1.21 -> 1.21.3-pmk.443
v1.22 -> 1.22.9-pmk.299
v1.23 -> 1.23.8-pmk.277
v1.24 -> 1.24.7-pmk.130

NOTE

Patches for v1.20 will be available soon.

If upgrade of cluster is not feasible proceed with the below workaround steps.

Workaround

• Edit the MetalLB controllerdeployment and change image location to quay:

Command

$ kubectl get deploy controller -n metallb-system -o yaml | grep image:
        image: metallb/controller:v0.10.2

• Update image to quay.io/metallb/controller:v0.10.2

Command

$ kubectl edit deploy controller -n metallb-system
deployment.apps/controller edited

$ kubectl get deploy controller -n metallb-system -o yaml | grep image:
        image: quay.io/metallb/controller:v0.10.2

• Edit the MetalLB speaker daemonset and the change image location to quay

Command

$ kubectl get daemonset speaker -n metallb-system -o yaml | grep image:
        image: metallb/speaker:v0.10.2

• Update image to quay.io/metallb/speaker:v0.10.2

Command

$ kubectl edit daemonset speaker -n metallb-system
daemonset.apps/speaker edited

$ kubectl get daemonset speaker -n metallb-system -o yaml | grep image:
        image: quay.io/metallb/speaker:v0.10.2

Additional Information

Reference https://github.com/metallb/metallb/issues/696#issuecomment-714652624.