# How to Upgrade PMK Cluster in Batches

## Problem

How to Upgrade PMK Cluster in Batches?

## Environment

* Platform9 Managed Kubernetes - v5.4 and Higher
* Qbert v4 API

## Procedure

1. Fetch Keystone authentication token in order to access the PMK REST API by following [Keystone Identity API](https://platform9.com/docs/kubernetes/keystone-identity-api).
2. Check if the PMK Cluster is available for an upgrade.

{% tabs %}
{% tab title="GET Cluster API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/clusters/<CLUSTER_UUID>
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example GET Cluster API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90 | jq | grep -e name -e uuid -e projectId -e kubeRoleVersion -e canUpgrade -e canMinorUpgrade -e minorUpgradeRoleVersion -e canPatchUpgrade -e patchUpgradeRoleVersion

  "name": "batch-upgrade-test",
  "uuid": "41596ba7-b6e5-4008-85e4-42080f43eb90",
  "canUpgrade": true,
  "projectId": "86f43d7fc7ce4287a6be1d861a9ac8b4",
  "kubeRoleVersion": "1.20.11-pmk.2064",
  "canMinorUpgrade": 1,
  "minorUpgradeRoleVersion": "1.21.3-pmk.72",
  "canPatchUpgrade": 0,
  "patchUpgradeRoleVersion": ""
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
**Info**

Cluster is currently on version *1.20.11-pmk.2064* and there is a minor upgrade available to version *1.21.3-pmk.72*.
{% endhint %}

3. Cluster Node Information

* Master Node

{% tabs %}
{% tab title="GET Master Node API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/nodes | jq '.[] | select(.clusterUuid=="<CLUSTER_UUID>" and .isMaster=='1')'
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example GET Master Node API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='1')' | grep -e uuid

  "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",
```

{% endtab %}
{% endtabs %}

* Worker Node

{% tabs %}
{% tab title="GET Worker Node API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://<DU_FQDN>/qbert/v4/<PROJECT_ID>/nodes | jq '.[] | select(.clusterUuid=="<CLUSTER_UUID>" and .isMaster=='0')'
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example GET Worker Node API" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='0')' | grep -e uuid

  "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",
  "uuid": "64e08702-041a-43fb-a708-00cb02112f19",
  "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",
```

{% endtab %}
{% endtabs %}

4. Performing Batch Cluster Upgrade

**Important**

As part of the Upgrade API with Batch Option, *only Worker Nodes* can/should be specified in the `batchUpgradeNodes` parameter.

The *Master Nodes* will always be upgraded first in a sequential manner followed by the worker nodes specified in the *batchUpgradeNodes* argument.

{% tabs %}
{% tab title="Upgrade API with Batch Option" %}

```bash
curl --request POST \
 --url https://platform9.io/qbert/v4/{project_uuid}/clusters/{uuid}/upgrade \
 --header "X-Auth-Token: {X-Auth-Token}" \
 --header "content-type: application/json" \
 --data '{
 "batchUpgradeNodes": [
  "{array[string]...}"
 ],
}'
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example Upgrade API with Batch Option for Minor Type" %}

```bash
curl --header "X-Auth-Token: $TOKEN" --header "content-type: application/json" --request POST https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90/upgrade/\?type\="minor" --data '{ "batchUpgradeNodes": ["522b7a45-b98a-4990-9482-826beab2dfbf"] }'

OK
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
**Note**

If Patch type cluster upgrade is available and that is desired, the same needs to be specified in the API call as `upgrade/\?type\="patch"`

`--header "content-type: application/json"` should be specified as shown above, otherwise the API call will upgrade all nodes in a sequential manner.
{% endhint %}

* The above example API upgraded the master nodes within the cluster first followed by the provided single worker node in batch.

{% tabs %}
{% tab title="Example Master Node Status" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='1')' | grep -e uuid -e actualKubeRoleVersion

  "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",
  "actualKubeRoleVersion": "1.21.3-pmk.72"
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example Worker Node Status" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90" and .isMaster=='0')' | grep -e uuid -e actualKubeRoleVersion

  "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",
  "actualKubeRoleVersion": "1.21.3-pmk.72",

  "uuid": "64e08702-041a-43fb-a708-00cb02112f19",
  "actualKubeRoleVersion": "1.20.11-pmk.2064",

  "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",
  "actualKubeRoleVersion": "1.20.11-pmk.2064",
```

{% endtab %}
{% endtabs %}

* Upgrade the remaining worker nodes as part of the next batch.

{% tabs %}
{% tab title="Example Upgrade API with Batch Option for Minor Type on Remaining Worker Nodes" %}

```bash
curl --header "X-Auth-Token: $TOKEN" --header "content-type: application/json" --request POST https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90/upgrade/\?type\="minor" --data '{ "batchUpgradeNodes": ["64e08702-041a-43fb-a708-00cb02112f19", "f3b0cbb0-efb9-407e-939a-21809421c376"] }'

OK
```

{% endtab %}
{% endtabs %}

* Post Upgrade Status

{% tabs %}
{% tab title="Example Cluster Status Post Upgrade" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/clusters/41596ba7-b6e5-4008-85e4-42080f43eb90 | jq | grep -e name -e uuid -e projectId -e kubeRoleVersion -e canUpgrade -e canMinorUpgrade -e minorUpgradeRoleVersion -e canPatchUpgrade -e patchUpgradeRoleVersion

  "name": "batch-upgrade-test",
  "uuid": "41596ba7-b6e5-4008-85e4-42080f43eb90",
  "canUpgrade": false,
  "projectId": "86f43d7fc7ce4287a6be1d861a9ac8b4",
  "kubeRoleVersion": "1.21.3-pmk.72",
  "canMinorUpgrade": 0,
  "minorUpgradeRoleVersion": "",
  "canPatchUpgrade": 0,
  "patchUpgradeRoleVersion": "",
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Example Node Status Post Upgrade" %}

```bash
curl --header "X-Auth-Token: $TOKEN" https://sanchit.platform9.horse/qbert/v4/86f43d7fc7ce4287a6be1d861a9ac8b4/nodes | jq '.[] | select(.clusterUuid=="41596ba7-b6e5-4008-85e4-42080f43eb90")' | grep -e uuid -e actualKubeRoleVersion

  "uuid": "522b7a45-b98a-4990-9482-826beab2dfbf",
  "actualKubeRoleVersion": "1.21.3-pmk.72",

  "uuid": "ba53cb28-14ab-4406-b352-9dd6a1314320",
  "actualKubeRoleVersion": "1.21.3-pmk.72",

  "uuid": "64e08702-041a-43fb-a708-00cb02112f19",
  "actualKubeRoleVersion": "1.21.3-pmk.72",

  "uuid": "f3b0cbb0-efb9-407e-939a-21809421c376",
  "actualKubeRoleVersion": "1.21.3-pmk.72",
```

{% endtab %}
{% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://platform9.com/kb/pmk/how-to/how-to-upgrade-cluster-in-batches.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
