PMK Release 5.8 Release Notes

PMK 5.8.0 Release Summary (Released 2023-05-15)

The Platform9 Managed Kubernetes (PMK) version 5.8 release is now generally available and continues to support Kubernetes v1.24. This release brings new features, enhancements and critical bug fixes to improve overall user experience and stability. This release also marks the Platform9 Managed KubeVirt as generally available.

Release Highlights

New Features

• Added the support for GP3 type EBS Volumes with IOPS & Throughput configuration on AWS Qbert Clusters.
• Added the option in UI for "kubernetes-internal-ip" as one of the Interface detection methods for Calico CNI.

Platform9 CLI

The 1.21 pf9ctl release is now available and can be installed by running the following command.

bash <(curl -sL https://pmkft-assets.s3-us-west-1.amazonaws.com/pf9ctl_setup)

Bug Fixes

Fixed Fixed a bug which prevented Sunpike CA rotation if the CA had an expiry of more than a year.

Fixed Fixed a bug in the UI which prevented the kube-apiserver flags to be added to apiserver configuration during cluster creation from the UI.

Fixed Fixed a bug in UI which prevented Add Topology Manager support option from being honored, for BareOS Cluster Creation from UI.

Fixed Fixed a bug in the UI which prevented the Self-service users with access to a particular namespace from being able to view resources in that namespace.

Package Updates

The following packed components have been upgraded in latest v1.24 Kubernetes version:

Please refer to the Managed Kubernetes Support Matrix for v5.8 to view all currently deployed or supported upstream component versions.

Known Issues

Known Issue External Clusters with name more than 63 characters will not be discoverable for Import operation. This is due to the limitation from K8s specs.

Known Issue A cluster that is renamed in PMK will not reflect the new name in ArgoCD. However, the cluster is still available under the older name in ArgoCD.

Known Issue Calico IPAM is only supported when using Calico CNI.

Known Issue EKS, AKS, or GKE Cluster Import “401 Unauthorized” Notification and Empty Dashboards.

If an AWS Cloud Provider is configured to import clusters without the correct identity being added to the target cluster, Platform9 will be unable to access the cluster.

It's important to note that if you have used a Cloud Provider to register an EKS, AKS, or GKE cluster that was created with IAM user credentials, which no longer have access to the EKS, AKS, or GKE K8s clusters, Platform9 will fail with an 401 unauthorized error until that IAM user is given access to the K8s cluster.

View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS, AKS, or GKE.

Known Issue Cluster upgrade attempt is blocked on UI post a cluster upgrade failure due to nodes being in a converging/not converged state.

Known Issue Hostpath-csi-driver installs to to default namespace only.

Known Issue Kubelet authorization mode is marked set to AlwaysAllow instead of Webhook.

Known Issue UI throws error when using SSO with Azure AD and passwordless logins.

Known Issue PMK Cloud provider created directly in Sunpike cannot be used to create qbert clusters. Qbert cloud providers will work to create both qbert and sunpike clusters. But cloud providers created directly in sunpike CANNOT be used to create qbert clusters. Please use the appropriate one based on your needs.

Known Issue Certificate generation fails if CA validity is less than the amount of TTL with which certificate is attempted to be generated in Vault

CAPI AWS & EKS clusters

Known Issue For CAPI based clusters, labels with special characters like - cannot be associated to the cluster. A known bug leads to only one part of the label being used when applying it to the cluster.

Known Issue For CAPI based clusters, downloading multiple kubeconfigs for a given user will invalidate the previous kubeconfigs of that user. Only the latest downloaded kubeconfig will be valid for that user account to use. User are recommended to use caution when sharing user accounts for such clusters.

Known Issue Users should not make changes to EKS clusters created using PMK, from AWS Console. Platform9 manages the lifecycle of the EKS control-plane and EKS nodes. Making any changes from AWS Console might result in undesired effects or render cluster in non-functional state.

Known Issue The Kubernetes dashboard is not accessible for CAPI based clusters by uploading the kubeconfig, because of an upstream issue where dashboard does not support OIDC-based kubeconfigs.

As a workaround, authenticate with the ID Token.

1. Download and open the kubeconfig of a cluster.
2. Copy the value of the id_token field.
3. In the dashboard, select "token" authentication and paste the value in the form.

Note: refreshing is not supported by the dashboard, this means you lose access after the token expires (10-20 min)

To refresh the ID token, simply run a kubectl command with it. kubectl will replace the ID token in the kubeconfig with a valid one if it has expired. Then afterwards follow the steps above again.

Known Issue EKS cluster created with with MachinePool / MachineDeployment type of worker node groups with desired node count can sometimes get stuck in “provisioned” state, for more than an hour after control plane is ready.

Current workarounds:

• Machine Deployment type Node Group: Scale down the affected node group to 0 and then scale it back up to desired count.
• Machine Pool type Node Group: Delete the affected node group and add new node group to the cluster.
• Create an EKS cluster with MachinePool / MachineDeployment with replica count as 0. Once the cluster is healthy change the replica count to desired value.

Known Issue In some instances a CAPI cluster when deleted during provisioning phase can get stuck into deleting phase. Please contact Platform9 support for possible options of resolution.

Known Issue CAPI Cluster creation in EKS with Private API Server Endpoint access is not supported. Will be added in next release.

KubeVirt

The Platform9 Managed KubeVirt is now Generally Available

The Platform9 Managed KubeVirt version 5.8 release is now generally available with support for Kubernetes v1.24 and Kubevirt v0.58.1, bringing new features, enhancements and bug fixes. Please refer Platform9 Managed KubeVirt 5.8 Release Notes for more information.