PMK Release 5.5 Release Notes

The Platform9 Managed Kubernetes (PMK) version 5.5 release is now available with support for Kubernetes 1.21. The 5.5 release aims to bring multifold improvements to the User Experience with a brand new UI and App-Switching capability. This release adds brand new features and services including ECO, ArgoCD-as-a-Service, Metal3, and Catapult. This release continues to upscale Platform9's commitment to provide features with deep Kubernetes integration, built on Open-Source technologies.

All clusters running Kubernetes 1.19 must be upgraded prior to upgrading to PMK 5.5

Kubernetes 1.19 has reach End of Life as of 2021-10-28. New clusters should be built on 1.21.

Kubernetes 1.20 has been deprecated as of 2022-02-28. New clusters should be built on 1.21.

PMK Release Highlights

Catapult Remote Monitoring

As part of the Platform9 5.5 release we have made significant changes to our remote monitoring capabilities, introducing Kubernetes specific metrics and alerts that provide an additional layer of managed protection.

Every user of Platform9 now has access to monitoring that remotely monitors the critical functions of their Kubernetes clusters. The new remote monitoring is known as Catapult and has been built leveraging 100% open source Kubernetes Native tooling. Catapult monitoring monitors the following items within each cluster:

• etcd
• API Server
• Nodes
• Pods
• Calico networking
• Platform9 Managed Add-ons Health
• Platform9 Connectivity

By collecting key metrics Platform9 is able to provide alerts on events such as etcd backup failures, API Server client errors, Node availability, Node memory pressure, Pod crashloopbackoff and more.

Prometheus will now also run in agent mode on each node attached to PMK. This prometheus agent will scrape metrics from a number of targets and remote write these metrics on the Prometheus running in PMK SaaS Management Plane. The Alert Manager running in PMK SaaS Management Plane will send alerts to the customers.

In total there are 56 built-in alerts that can be configured to notify your team over email, webhook, Slack and ServiceNow.

A remarkable new user experience.

PMK version 5.5 has a rebuilt WebApp that provides a significantly improved user experience. The following screenshots demonstrate various sections of the new user interface.

New Theming Capability

We added the ability to choose different themes to customize the appearance. Modifying the themes allow users to maximize productivity while enjoying an increased visual experience. These themes are divided into four main views.

• Light: The light theme is a clean looking motif that is very easy on the eyes and useful in brighter environments.
• Default: The default theme is not as bright as the light theme, but it is a low enough contrast to offset the lower contrast dark and ultra dark themes.
• Dark: Dark themes help reduce the brightness level cast by device screens, while meeting the minimal color contrast ratios needed to reduce eye strain. Adjusting the brightness down to a lower level better matches low light conditions. This facilitates improved screen usage in darker environments, which also conserves battery power.
• Ultra Dark: This theme uses an even darker color than the above dark theme, making it even easier on the eyes or excellent for a higher contrast option.

Application Switcher

With the arrival of PMK version 5.5, we have included a new App Switcher, which allows users the ability to maneuver seamlessly across the different services in the PMK UI.

Deep-dive Cloud Native App Dashboards

In our continual effort to make Kubernetes easier to manage for both operators and developers, PMK 5.5 adds a complete set of deep-dive dashboards for working with cloud native applications. This includes: Pods, Deployments, Services, Stateful Sets, Replica Sets, Daemon Sets, Jobs & CronJobs sections now have dedicated views which allow users to:

• All resources are automatically linked to keep data in context
• Better visualize cluster configurations.
• Instantly access events and logs

Each in-depth dashboard provides the ability to review and quickly modify an object's YAML file for faster updates, and delivers the improved ability to troubleshoot issues.

EKS, AKS and GKE Management using ECO

With 5.5 release we are enhancing our external cluster management with the introduction of the External Cluster Operator (ECO).

ECO is installed into EKS, AKS and GKE clusters, creating a secure outbound tunnel that connects the cluster to the SaaS Management Plane.

Once connected, ECO enables users to leverage Platform9 Managed Add-ons, such as in-cluster monitoring, deploy apps using ArgoCD to the Helm Service and quickly troubleshoot applications through the workload dashboards.

Platform9 CLI

The 1.14 pf9ctl release is now available and can be installed by running the following command.

Added Cluster bootstrap command

Added Cluster delete command

Added New node commands

• detach-node
• authorize-node
• deauthorize-node
• decommission-node

Added Unattended mode support with --no-prompt flag to make pf9ctl more automation friendly.

Enhanced Attach node command to also accept cluster UUID in place of cluster name while attaching a node to the cluster.

Enhanced Remote node support for all node commands.

Fixed Node command related bug fixes.

ArgoCD as a Service [Early access]

(Link Removed)

Metal3

Platform9 Managed Bare Metal Release Notes

Enhancements & Updates

Added Added ability to customize ETCD via the PMK Qbert APIs using custom environment variables.

Added Added DNS-autoscalar as part of the Coredns add-on, with preventSinglePointFailure set to true by default, which ensures two CoreDNS replicas on a multi-node cluster.

Fixed Fixed a bug because of which upgrade of Container Runtime from Docker to Containerd did not work, on cluster upgrade from Kubernetes version 1.20 to 1.21.

Fixed Fixed a bug which re-enabled the disabled add-ons post cluster upgrade.

Fixed Fixed a bug that caused PMK components to fail to start post node reboot on clusters with Containerd Runtime on Kubernetes Version 1.21

Fixed Fixed a bug that failed rotation of access keys for AWS Cloud Provider with existing Kubernetes clusters via PMK UI.

Fixed Fixed a bug that caused cluster creation failure on PMK instances having 100 or more nodes onboarded.

Fixed Fixed a bug that caused the Add-on Operator pod to get killed due to insufficient memory.

Fixed Fixed missing documentation for Contained Runtime in Qbert API docs.

Fixed Fixed a bug causing failure of monitoring add-on on cluster upgrades from Kubernetes version 1.19 to higher versions.

Fixed Fixed a bug that caused the failure of monitoring add-on and unavailability of Grafana links on UI, on cluster upgrades from the Kubernetes version 1.20 to 1.21.

Fixed Fixed a bug raising wrong alerts marked for PMK components, where the alerts were for user workloads.

Fixed Fixed a bug because of which ETCD Backup interval input in minutes did not work.

Fixed Fixed a bug that would cause nodes set as unscheduled to automatically reset to schedule after a reboot.

New New Added Add-on management in the PMK Cluster edit flow.

New New: Added a container details screen

New New Added Daemon Sets in the Workloads in UI.

New New Added ReplicaSets in the Workloads in UI.

New New Added Stateful Sets in the Workloads in UI.

New New Added Jobs and CronJobs in the Workloads in UI.

New New Added new Navigation and Theme to improve User Experience and provide an IDE experience in the UI.

New New Added Ideas section in the Contact Us page for users.

New New Added Kubernetes Node status on Cluster and Nodes pages under Infrastructure tab in UI.

New New Added allow users to create a Read-Only user.

New New Added External Cluster Operator for Imported Clusters.

New New Added a feature allowing users to Import and Save/Export the drift analysis.

New New Added Creating and Enabling Keystone Groups and User Associations for all Users.

Enhanced Updated the UI design for Workloads for a better User Experience.

Enhanced Updated Profile Engine is now enabled for all clusters by default.

Fixed Fixed a bug in UI that caused a discrepancy in the repository names in the edit application flow when two different applications are deployed with the same name in the same cluster and namespace.

Fixed Fixed a bug in UI that caused persistent failure in upgrade retry of a failed upgrade when upgrading the Kubernetes cluster from version 1.20 to 1.21 and Container Runtime from Docker to Containerd.

Fixed Fixed a bug in UI that allowed enabling of Multi-Factor Authentication for users already using Single Sign-on.

Fixed Fixed a bug in the UI because of which an incorrect ETCD backup path was shown on the edit cluster page.

Fixed Fixed a bug because of which Minimum and Maximum nodes options were not visible in Scale worker action when cluster autoscalar is enabled.

Fixed Fixed a bug in UI showing the wrong status for pending Persistent Volume Claims.

Fixed Fixed a bug in UI that caused Cloud Provider defaults to be global instead of being scoped to accounts.

Fixed Fixed an issue on the Node Health dashboard that would cause selected nodes to disappear from the dashboard.

Fixed Fixed bug in UI because of which the 'No Cluster' filer on the Nodes dashboard could not be clicked.

Fixed Fixed a bug in UI that caused inconsistencies in the MFA enablement.

Package Updates

The following packed components have been upgraded:

1. calico: “v3.18.1”
2. casaws: “1.14.8"
3. casazure: “1.13.8”
4. cni: “v0.9.0"
5. coredns: “1.8.0”
6. dashboard: “2.0.1"
7. etcd: “v3.4.14”
8. flannel: “v0.14.0"
9. kubevirtaddon: “0.45.0"
10. luigi: “0.3.0"
11. metallb: “0.10.2”
12. metricsserver: “0.5.0"
13. monitoring: “0.46.0”
14. profileagent: “2.0.0"

Please refer to the Managed Kubernetes Support Matrix for v5.5 to view all currently deployed or supported upstream component versions.

Known Issues

Known Issue Calico IPAM is only supported when using Calico CNI.

Known Issue EKS, AKS, or GKE Cluster Import “401 Unauthorized” Notification and Empty Dashboards.

If an AWS Cloud Provider is configured to import clusters without the correct identity being added to the target cluster, Platform9 will be unable to access the cluster.

It's important to note that if you have used a Cloud Provider to register an EKS, AKS, or GKE cluster that was created with IAM user credentials, which no longer have access to the EKS, AKS, or GKE K8s clusters, Platform9 will fail with an 401 unauthorized error until that IAM user is given access to the K8s cluster.

View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS, AKS, or GKE.

Known Issue Registry Mirror configuration is not available for Containerd runtime based PMK clusters.

Known Issue Docker cache is not deleted when PMK clusters are migrated from docker to containerd.

Current workaround is to delete /var/lib/docker manually.

Known Issue PMK certificate generation fails if CA validity is less the amount of TTL with which certificate is attempted to be generated in Vault.

Known Issue Cluster upgrade attempt is blocked on UI post a cluster upgrade failure due to nodes being in a converging/not converged state.