How To Add/Modify Existing MetalLB Configuration via API

Problem

How To Add/Modify Existing MetalLB Configuration via API on a PMK cluster?

Environment

• Platform9 Managed Kubernetes

• AddOn Operator v4.0.0 & MetalLB v0.9.7 and Above

Procedure

Verify the pf9-addon-operator version on the cluster.

pf9-addon-operator Image

# kubectl describe deployment pf9-addon-operator -n pf9-addons | grep -i image
    Image:      platform9/pf9-addon-operator:5.0.0

Info

One can add/modify the set/existing MetalLB configuration as desired by following the steps below.

Note: For Cluster running pf9-addon-operator version below 4.0.0, set the watch attribute in the MetalLB ClusterAddOn object to false and then make changes to the MetalLB configmap directly so that the AddOn operator will not revert the changes made to the configmap. Note that the same will be overwritten post cluster upgrade.

1. Fetch the authentication TOKEN. Keystone Identity API: This link describes the process for getting a Keystone authentication token in order to access the PMK REST API.

2. API to retrieve currently set MetalLB ClusterAddon object details.

curl -X GET -H "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/sunpike/apis/sunpike.platform9.com/v1alpha2/namespaces/default/clusteraddons/<CLUSTER_UUID>-metallb

Info

Replace the placeholders <DU_FQDN>, <PROJECT_ID> and the <CLUSTER_UUID> with their appropriate values.

• MetalLB ClusterAddon object here with single address pool range.

Example GET API

curl -X GET -H "X-Auth-Token: $TOKEN" https://cse.platform9.io/qbert/v4/b855dca4be144f089b68dcff7c39f820/sunpike/apis/sunpike.platform9.com/v1alpha2/namespaces/default/clusteraddons/8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1446  100  1446    0     0   1948      0 --:--:-- --:--:-- --:--:--  1972
{
  "kind": "ClusterAddon",
  "apiVersion": "sunpike.platform9.com/v1alpha2",
  "metadata": {
    "name": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb",
...
...
  "spec": {
    "clusterID": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7",
    "version": "0.10.2",
    "type": "metallb",
    "override": {
      "params": [
        {
          "name": "MetallbIpRange",
          "value": "10.128.147.247-10.128.147.249"
        }
      ]
    },
    "watch": true
  },
  "status": {
    "phase": "Installed",
    "healthy": true,
    "lastChecked": null
  }
}

• Related Existing ConfigMap.

Example of Existing ConfigMap

$ kubectl describe configmap config -n metallb-system
Name:         config
Namespace:    metallb-system
Labels:       <none>
Annotations:  <none>

Data
====
config:
----
address-pools:
- name: default
  protocol: layer2
  addresses:
  - 10.128.147.247-10.128.147.249

Events:  <none>

1. Create a new JSON spec for MetalLB.

Info

Replace the values for clusterID & sunpike.pf9.io/cluster with required CLUSTER_UUID.

The name attribute in the spec must be in the format CLUSTER_UUID-metallb.

Spec Example

# cat metallbaddon.json
{
  "apiVersion": "sunpike.platform9.com/v1alpha2",
  "kind": "ClusterAddon",
  "metadata": {
    "labels": {
      "sunpike.pf9.io/cluster": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7",
      "type": "metallb"
    },
    "name": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb",
    "namespace": "default"
  },
  "spec": {
    "clusterID": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7",
    "override": {
      "params": [
        {
          "name": "base64EncMetallbConfig",
          "value": "YWRkcmVzcy1wb29sczoKLSBuYW1lOiBkZWZhdWx0CiAgcHJvdG9jb2w6IGxheWVyMgogIGFkZHJlc3NlczoKICAgLSAxMC4xMjguMTQ3LjI0Ny0xMC4xMjguMTQ3LjI0OAotIG5hbWU6IHBvb2wKICBwcm90b2NvbDogbGF5ZXIyCiAgYWRkcmVzc2VzOgogICAtIDEwLjEyOC4xNDcuMjQ5LTEwLjEyOC4xNDcuMjUw"
        }
      ]
    },
    "type": "metallb",
    "version": "0.10.2",
    "watch": true
  }
}

Example Encoded string value for parameter base64EncMetallbConfig

YWRkcmVzcy1wb29sczoKLSBuYW1lOiBkZWZhdWx0CiAgcHJvdG9jb2w6IGxheWVyMgogIGFkZHJlc3NlczoKICAgLSAxMC4xMjguMTQ3LjI0Ny0xMC4xMjguMTQ3LjI0OAotIG5hbWU6IHBvb2wKICBwcm90b2NvbDogbGF5ZXIyCiAgYWRkcmVzc2VzOgogICAtIDEwLjEyOC4xNDcuMjQ5LTEwLjEyOC4xNDcuMjUw

translates to

Example Decoded string value for base64EncMetallbConfig

address-pools:
- name: default
  protocol: layer2
  addresses:
   - 10.128.147.247-10.128.147.248
- name: pool
  protocol: layer2
  addresses:
   - 10.128.147.249-10.128.147.250

Note

In the above example, we are passing base64EncMetallbConfig string which includes Multiple Named Address Pools which have different IP allocations than the original MetallbIpRange.

1. Patching the existing MetalLB ClusterAddon object with the above-mentioned new JSON spec metallbaddon.json.

PATCH API

curl -X PATCH -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/merge-patch+json" -H "Accept: application/json" -d "@metallbaddon.json" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/sunpike/apis/sunpike.platform9.com/v1alpha2/namespaces/default/clusteraddons/<CLUSTER_UUID>-metallb

Example Patch API

# curl -X PATCH -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/merge-patch+json" -H "Accept: application/json" -d "@metallbaddon.json" https://cse.platform9.io/qbert/v4/b855dca4be144f089b68dcff7c39f820/sunpike/apis/sunpike.platform9.com/v1alpha2/namespaces/default/clusteraddons/8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb

...

• Updated MetalLB ClusterAddon object

Example GET API Post Update

curl -X GET -H "X-Auth-Token: $TOKEN" https://cse.platform9.io/qbert/v4/b855dca4be144f089b68dcff7c39f820/sunpike/apis/sunpike.platform9.com/v1alpha2/namespaces/default/clusteraddons/8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb | jq

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1825  100  1825    0     0   2351      0 --:--:-- --:--:-- --:--:--  2382
{
  "kind": "ClusterAddon",
  "apiVersion": "sunpike.platform9.com/v1alpha2",
  "metadata": {
    "name": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7-metallb",
...
...
  "spec": {
    "clusterID": "8c3bd9bf-951d-4d60-88dc-aa42c4f548d7",
    "version": "0.10.2",
    "type": "metallb",
    "override": {
      "params": [
        {
          "name": "base64EncMetallbConfig",
          "value": "YWRkcmVzcy1wb29sczoKLSBuYW1lOiBkZWZhdWx0CiAgcHJvdG9jb2w6IGxheWVyMgogIGFkZHJlc3NlczoKICAgLSAxMC4xMjguMTQ3LjI0Ny0xMC4xMjguMTQ3LjI0OAotIG5hbWU6IHBvb2wKICBwcm90b2NvbDogbGF5ZXIyCiAgYWRkcmVzc2VzOgogICAtIDEwLjEyOC4xNDcuMjQ5LTEwLjEyOC4xNDcuMjUw"
        }
      ]
    },
    "watch": true
  },
  "status": {
    "phase": "Installed",
    "healthy": true,
    "lastChecked": null
  }
}

• ConfigMap Updated After the PATCH Request.

Example of Updated ConfigMap

$ kubectl describe configmap config -n metallb-system
Name:         config
Namespace:    metallb-system
Labels:       <none>
Annotations:  <none>

Data
====
config:
----
address-pools:
- name: default
  protocol: layer2
  addresses:
   - 10.128.147.247-10.128.147.248
- name: pool
  protocol: layer2
  addresses:
   - 10.128.147.249-10.128.147.250

Events:  <none>

Additional Information

• Within PMK, currently while creating a cluster you can provide multiple address pools for MetalLB. *Reference: *Create a cluster using auto-deploy or manual mode